__ __ / \ / \ ______ _ _ __ __ __ / /\ / /\ \ / __ \ | | | | / \ \ \ / / / / / / \ \ | | | | | | | | / /\ \ \ / \ \ \ \ / / | |__| | | |__| | / ____ \ | | \ \/ \ \/ / \_ ___/ \____/ /_/ \_\ |_| \__/ \__/ \ \__ \___\ by Red Hat Build, Store, and Distribute your Containers Startup timestamp: Tue Aug 8 05:45:23 UTC 2023 Running all default registry services without migration Running init script '/quay-registry/conf/init/certs_install.sh' Installing extra certificates found in /quay-registry/conf/stack/extra_ca_certs directory Running init script '/quay-registry/conf/init/copy_config_files.sh' Running init script '/quay-registry/conf/init/d_validate_config_bundle.sh' Validating Configuration time="2023-08-08T05:45:24Z" level=debug msg="Validating AccessSettings" time="2023-08-08T05:45:24Z" level=debug msg="Validating ActionLogArchiving" time="2023-08-08T05:45:24Z" level=debug msg="Validating AppTokenAuthentication" time="2023-08-08T05:45:24Z" level=debug msg="Validating BitbucketBuildTrigger" time="2023-08-08T05:45:24Z" level=debug msg="bitbucket response: 400, {\"error\": \"invalid_grant\", \"error_description\": \"The specified code is not valid.\"}" time="2023-08-08T05:45:24Z" level=debug msg="Validating BuildManager" time="2023-08-08T05:45:24Z" level=debug msg="Validating Database" time="2023-08-08T05:45:24Z" level=debug msg="Scheme: postgresql" time="2023-08-08T05:45:24Z" level=debug msg="Host: quayregistry-quay-database:5432" time="2023-08-08T05:45:24Z" level=debug msg="Db: quayregistry-quay-database" time="2023-08-08T05:45:24Z" level=debug msg="Params: " time="2023-08-08T05:45:24Z" level=debug msg="Including params " time="2023-08-08T05:45:24Z" level=debug msg="Pinging database at postgresql://quayregistry-quay-database:2UWTsTv2QdQ6mc00jJgNWEJ8XiY1gI2X4re8FqDQpfkq2v583JBpb4gythFSV9EtzeStO4mPKnWJ9QB3@quayregistry-quay-database:5432/quayregistry-quay-database" plpgsql pg_trgm time="2023-08-08T05:45:24Z" level=debug msg="Validating DistributedStorage" time="2023-08-08T05:45:25Z" level=debug msg="Validating ElasticSearch" time="2023-08-08T05:45:25Z" level=debug msg="Validating Email" time="2023-08-08T05:45:25Z" level=debug msg="Validating GitHubBuildTrigger" time="2023-08-08T05:45:25Z" level=debug msg="github response: 200, \n\n\n\n\n\n\n\n \n \n \n \n \n \n \n \n\n \n\n \n \n \n \n \n \n \n\n\n\n\n \n\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n\n GitHub: Let’s build from here · GitHub\n\n\n\n \n\n \n \n\n\n \n\n\n\n \n\n \n \n\n \n \n\n \n \n \n \n \n\n\n\n \n\n \n\n\n\n\n \n\n \n\n \n\n \n \n \n \n \n \n \n \n \n \n\n\n\n \n\n\n\n \n\n \n\n\n \n \n \n \n\n \n\n \n \n \n\n \n \n \n\n\n\n \n \n\n\n \n\n \n\n \n\n \n \n \n\n\n\n\n\n \n\n \n\n \n
\n \n\n\n
\n Skip to content\n \n \n \n \n\n\n \n\n \n\n \n\n\n\n
\n \n\n
\n
\n \n \n \n\n \n\n
\n \n Sign up\n \n
\n\n
\n \n
\n
\n\n\n
\n
\n \n\n
\n \n\n\n\n \n \n
\n \n \n\n
\n Search or jump to...\n
\n \n\n
\n \n\n \n\n \n
\n \n

Search code, repositories, users, issues, pull requests...

\n
\n \n
\n
\n \n
\n \n
\n \n \n \n\n \n
\n
\n
\n
\n \n
\n
\n Clear\n \n \n\n
\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n \n
\n
\n
\n\n
\n \n Search syntax tips\n
\n
\n
\n
\n\n
\n
\n
\n \n
\n \n
\n \n
\n
\n
\n

\n Provide feedback\n

\n
\n
\n \n
\n
\n
\n
\n

We read every piece of feedback, and take your input very seriously.

\n \n \n \n
\n
\n \n
\n
\n\n \n \n
\n \n
\n
\n
\n

\n Saved searches\n

\n

Use saved searches to filter your results more quickly

\n
\n
\n \n
\n
\n
\n
\n\n \n\n
\n
\n
\n\n
\n
\n \n
\n
\n \n
\n\n\n\n
\n \n Sign in\n \n
\n\n \n Sign up\n \n
\n
\n
\n
\n\n\n \n \n\n
\n\n\n\n\n\n\n\n\n
\n\n\n\n\n\n \n
\n\n\n \n \n\n\n\n\n\n\n \n
\n\n\n\n
\n
\n \n \n \n\n \"\"\n \n
\n\n
\n
\n \n\n
\n
\n \"\"\n\n
\n
\n \n \n\n \n
\n
\n\n
\n
\n\n
\n
\n \n
\n \n \n \n \"\"\n\n
\n
Introducing GitHub Copilot X
\n
Your AI pair programmer is leveling up
\n
\n\n
\n \n\n
\n
\n\n
\n\n

\n Let’s build from here\n

\n\n

\n The AI-Powered Developer Platform to Build, Scale, and Deliver Secure Software\n

\n\n
\n
\n
\n
\n \n
\n
\n \n
\n
\n \n \n
\n
\n
\n\n \n Start a free enterprise trial\n \n \n\n\n
\n\n
\n

Trusted by the world’s leading organizations ↘︎

\n\n
\n \"Stripe\n \"Pinterest\n
\n \"KPMG\n
\n \"Mercedes-Benz\n
\n \"P&G\n \"Telus\n
\n
\n
\n
\n
\n
\n\n
\n
\n
\n \n\n \n\n
\n Productivity\n Collaboration\n Security\n
\n\n
\n \n Start a free enterprise trial\n \n \n\n\n \n Sign up for GitHub\n \n \n\n\n
\n
\n
\n
\n\n
\n
\n \n
\n
\n
\n \n \n\n \n
\n\n
\n
\n\n
\n

Productivity

\n

\n Accelerate high-quality software development.\n Our AI-powered platform drives innovation with tools that boost developer velocity.\n

\n
\n
\n\n
\n
\n
\n \n \n\n \n \n\n
\n
\n \n \n\n
mona-github-github-g59jpq2w5w7.github.dev
\n \n \n\n
\n
\n \n \n\n
\n
\n\n
\n
\n \n \n\n
\n\n
\n
\n
    \n
  • index.html
    • \n
  • script.js
    • \n
  • package.json
    • \n
\n\n
\n
\n
\n
1
\n
2
\n
3
\n
4
\n
5
\n
6
\n
7
\n
8
\n
9
\n
10
\n
11
\n
12
\n
\n\n
\n 
<div class="position-absolute width-full color-bg-default" style="bottom: -4rem;">
\n 
  <div class="container-xl p-responsive">
\n 
    <div class="d-flex flex-justify-center flex-lg-justify-end color-bg-default">
\n 
      <div class="col-8 col-sm-7 col-md-6 col-lg-5 position-relative z-2 right-lg-n12 events-none">
\n 
        <picture>
\n 
          <source srcset="astro-mona.webp" type="image/webp">
\n 
          <img src="astro-mona.svg" width="960" height="967" class="home-astro-mona width-full position-absolute bottom-0 height-auto" alt="Mona looking at GitHub activity across the globe">
\n 
        </picture>
\n 
      </div>
\n 
    </div>
\n 
  </div>
\n 
</div>
\n\n
\n
\n\n\n
\n
\n\n
\n
    \n
  • Terminal
    • \n
  • Output
    • \n
  • Problems
    • \n
  • Debug Console
    • \n
\n\n
\n
\n [09:43:36]\n Starting\n 'watch-extension:vscode-api-tests'\n ...\n
\n
\n [09:43:36]\n Finished\n 'clean-extension:typescript-language-features'\n after\n 248 ms\n
\n
\n [09:43:36]\n Starting\n 'watch-extension:typescript-language-features'\n ...\n
\n
\n [09:43:36]\n Finished\n 'clean-extension:php-language-features'\n after\n 384 ms\n
\n
\n [09:43:36]\n Starting\n 'watch-extension:php-language-features'\n ...\n
\n
\n [09:43:40]\n Finished\n 'clean-extension:html-language-features-server'\n after\n 4.66 s\n
\n
\n [09:43:40]\n Starting\n 'watch-extension:html-language-features-server'\n ...\n
\n
\n [09:43:43]\n Finished\n 'clean-client'\n after\n 7.33 s\n
\n
\n [09:43:43]\n Starting\n 'watch-client'\n ...\n
\n
\n
\n
\n\n
\n
\n
\n \n
\n
\n \n
\n
\n \n
\n
\n\n
\n \n \n \n \"Planet\n\n
\n
\n
\n
\n\n\n
\n \"\"\n\n
\n
\n \"\"\n
\n\n
\n
\n
\n
\n \n \n \n \n

GitHub Codespaces offers a complete dev environment in seconds, so you can code, build, test, and open pull requests from any repo anywhere.

\n \n \n Check out GitHub Codespaces \n\n\n \n
\n
\n\n
\n
\n \n
\n
\n \"\"\n \"\"\n
\n
\n \"\"\n
\n
\n\n
\n
\n
\n
\n
\n
\n Did you know?\n
\n\n

22% increase

\n

in developer productivity
after three years with GitHub1

\n
\n
\n
\n
\n\n
\n \n
\n
\n

GitHub Copilot is your AI pair programmer that empowers you to complete tasks 55% faster by turning natural language prompts into coding suggestions.

\n\n \n
\n \n
\n
\n \n\n \n
\n \n
\n\n
\n
\n
\n
1
\n
2
\n
3
\n
4
\n
5
\n
6
\n
7
\n
8
\n
\n\n
\n 
import matplotlib.pyplot as plt
\n
\n 
def draw_scatterplot(x_values, y_values):
\n 
    plt.scatter(x_values, y_values, s=20)
\n 
    plt.title("Scatter Plot")
\n 
    plt.xlabel("x values")
\n 
    plt.ylabel("y values")
\n 
    plt.show()
\n\n
\n \n\n Copilot\n
\n
\n
\n\n
\n
\n\n
\n
\n
\n\n
\n
\n
\n\n
\n \n
\n \n
\n
\n

GitHub Actions automates your build, test, and deployment workflow with simple and secure CI/CD.

\n\n \n
\n
\n \n \n \"\"\n
\n \n\n
\n
\n\n
\n\n
\n \n
\n
\n

GitHub Mobile fits your projects in your pocket, so you never miss a beat while on the go.

\n\n \n
\n
\n \n \n \"\"\n
\n \n\n
\n
\n\n
\n\n
\n\n
\n
\n
\n\n
\n
\n\n
\n
\n \n
\n
\n
\n \n \n\n \n
\n\n
\n
\n\n
\n

Collaboration

\n

\n Supercharge collaboration.\n We provide unlimited repositories, best-in-class version control, and the world’s most powerful open source community—so your team can work more efficiently together.\n

\n
\n
\n\n \n \n \n \"Illustration\n\n\n
\n \"\"\n\n
\n
\n \"\"\n
\n\n
\n
\n
\n
\n \n \n \n \n

GitHub Issues and GitHub Projects supply flexible project management tools that adapt to your team alongside your code.

\n \n \n Explore GitHub Issues \n\n\n \n
\n
\n\n
\n
\n \n
\n
\n \"\"\n
\n
\n\n
\n
\n
\n
\n
\n
\n Did you know?\n
\n\n

80% reduction

\n

in onboarding time with GitHub1

\n
\n
\n
\n
\n\n
\n \n
\n \n
\n
\n

GitHub Discussions create space to ask questions and have open-ended conversations.

\n\n \n
\n
\n \n \n \"\"\n
\n \n\n
\n
\n\n
\n\n
\n \n
\n
\n

Pull requests allow real-time communication and collaboration about code changes.

\n\n \n
\n
\n \n \n \"\"\n
\n \n\n
\n
\n\n
\n\n
\n\n \n
\n
\n

GitHub Sponsors lets you support your favorite open source maintainers and projects.

\n\n \n
\n \n
\n
\n
\n \n \"Evan\n
Evan You
\n \n
\n
\n \n \"Nick\n
Nick DeJesus
\n \n
\n
\n \n \"Nikema\n
Nikema
\n \n
\n
\n \n \"kazuya\n
kazuya kawaguchi
\n \n
\n
\n \n \"CommandPost\n
CommandPost
\n \n
\n
\n \n \"Samuel\n
Samuel
\n \n
\n
\n \n \"chaynHQ\n
chaynHQ
\n \n
\n
\n \n \"sindresorhus\n
sindresorhus
\n \n
\n
\n \n \"ESLint\n
ESLint
\n \n
\n
\n
\n\n
\n
\n\n
\n
\n
\n\n
\n
\n
\n
\n \n
\n
\n
\n \n \n\n \n
\n\n
\n
\n\n
\n

Security

\n

\n Embed security into the developer workflow.\n With GitHub, developers can secure their code in minutes and organizations can automatically comply with regulations.\n

\n
\n
\n\n \n
\n
\n
cmake.yml
\n on: push\n
\n\n
\n
\n
\n
    \n
  • \n
    \n \n \n\n Build\n
    \n 1m 21s\n
    • \n
\n
\n
\n\n
\n
\n
\n
\n
\n\n
\n
\n Steps\n
\n
\n
    \n
  • \n
    \n \n \n\n Initialize CodeQL\n
    \n 1m 42s\n
    • \n
  • \n
    \n \n \n\n Autobuild\n
    \n 1m 24s\n
    • \n
  • \n
    \n \n \n\n Perform CodeQL Analyses\n
    \n 1m 36s\n
    • \n
\n
\n
\n
\n
\n
\n \n\n
\n \"\"\n\n
\n
\n \"\"\n
\n\n
\n
\n
\n
\n \n \n \n \n

GitHub Advanced Security lets you gain visibility into your security posture, respond to threats proactively, and ship secure applications quickly.

\n \n \n Get GitHub Advanced Security \n\n\n \n
\n
\n\n
\n
\n \n
\n
\n \"\"\n
\n
\n\n
\n
\n
\n
\n
\n
\n Did you know?\n
\n\n

56 million projects

\n

fixed vulnerabilities with GitHub2

\n
\n
\n
\n
\n\n
\n \n
\n
\n

Secret scanning automatically looks for partner patterns and prevents fraudulent use of accidentally committed secrets.

\n\n \n
\n
\n \n \n \"\"\n
\n \n\n
\n
\n\n
\n\n
\n \n
\n \n
\n
\n

Dependabot makes it easy to find and fix vulnerable dependencies in your supply chain.

\n\n \n
\n
\n \n \n \"\"\n
\n \n\n
\n
\n\n
\n\n
\n \n
\n
\n

Code scanning is GitHub’s static code analysis tool that helps you remediate issues in your code.

\n\n \n
\n
\n \n \n \"\"\n
\n \n\n
\n
\n\n
\n\n
\n\n
\n
\n\n
\n \n \n
\n
\n
\n
\n \n \n \n \n

The place for anyone from anywhere to\u00a0build\u00a0anything

\n

Whether you’re scaling your startup or just learning how to code, GitHub is your home. Join the world’s largest developer platform to build the innovations that empower humanity. Let’s\u00a0build from\u00a0here.

\n \n
\n\n
\n \n Sign up for GitHub\n \n \n\n\n \n Start a free enterprise trial\n \n \n\n\n
\n\n
\n\n
\n
\n\n
\n\n
\n
\n
\n \n \n \n\n \n\n
\n
\n\n \n
\n\n\n \n \n \n \"\"\n\n\n
\n \"Mona\n
\n\n \n\n\n \n
\n
\n
    \n
  • 1 The Total Economic Impact™ Of GitHub Enterprise Cloud and Advanced Security, a commissioned study conducted by Forrester Consulting, 2022. Results are for a composite organization based on interviewed customers.
    • \n
  • 2 GitHub, Octoverse 2022 The state of open source software.
    • \n
\n
\n\n
\n\n \n\n \n\n\n\n\n \n\n \n\n
\n
\n
\n
\n\n \n\n\n\n\n\n \n\n
\n \n\n\n" time="2023-08-08T05:45:25Z" level=debug msg="Validating GitHubLogin" time="2023-08-08T05:45:25Z" level=debug msg="Validating GitLabBuildTrigger" time="2023-08-08T05:45:25Z" level=debug msg="Validating GoogleLogin" time="2023-08-08T05:45:25Z" level=debug msg="Validating HostSettings" time="2023-08-08T05:45:25Z" level=debug msg="Validating JWTAuthentication" time="2023-08-08T05:45:25Z" level=debug msg="Validating LDAP" time="2023-08-08T05:45:25Z" level=debug msg="Validating OIDC" time="2023-08-08T05:45:25Z" level=debug msg="Validating QuayDocumentation" time="2023-08-08T05:45:25Z" level=debug msg="Validating Redis" time="2023-08-08T05:45:25Z" level=debug msg="Address: quayregistry-quay-redis:6379" time="2023-08-08T05:45:25Z" level=debug msg="Username: " time="2023-08-08T05:45:25Z" level=debug msg="Password Len: 0" time="2023-08-08T05:45:25Z" level=debug msg="Ssl: " time="2023-08-08T05:45:25Z" level=debug msg="Address: quayregistry-quay-redis:6379" time="2023-08-08T05:45:25Z" level=debug msg="Username: " time="2023-08-08T05:45:25Z" level=debug msg="Password Len: 0" time="2023-08-08T05:45:25Z" level=debug msg="Ssl: " time="2023-08-08T05:45:25Z" level=debug msg="Validating RepoMirror" time="2023-08-08T05:45:25Z" level=debug msg="Validating SecurityScanner" time="2023-08-08T05:45:25Z" level=debug msg="Validating TeamSyncing" time="2023-08-08T05:45:25Z" level=debug msg="Validating TimeMachine" time="2023-08-08T05:45:25Z" level=debug msg="Validating UserVisibleSettings" +------------------------+-------+--------+ | Field Group | Error | Status | +------------------------+-------+--------+ | AccessSettings | - | 🟢 | +------------------------+-------+--------+ | ActionLogArchiving | - | 🟢 | +------------------------+-------+--------+ | AppTokenAuthentication | - | 🟢 | +------------------------+-------+--------+ | BitbucketBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | BuildManager | - | 🟢 | +------------------------+-------+--------+ | Database | - | 🟢 | +------------------------+-------+--------+ | DistributedStorage | - | 🟢 | +------------------------+-------+--------+ | ElasticSearch | - | 🟢 | +------------------------+-------+--------+ | Email | - | 🟢 | +------------------------+-------+--------+ | GitHubBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | GitHubLogin | - | 🟢 | +------------------------+-------+--------+ | GitLabBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | GoogleLogin | - | 🟢 | +------------------------+-------+--------+ | HostSettings | - | 🟢 | +------------------------+-------+--------+ | JWTAuthentication | - | 🟢 | +------------------------+-------+--------+ | LDAP | - | 🟢 | +------------------------+-------+--------+ | OIDC | - | 🟢 | +------------------------+-------+--------+ | QuayDocumentation | - | 🟢 | +------------------------+-------+--------+ | Redis | - | 🟢 | +------------------------+-------+--------+ | RepoMirror | - | 🟢 | +------------------------+-------+--------+ | SecurityScanner | - | 🟢 | +------------------------+-------+--------+ | TeamSyncing | - | 🟢 | +------------------------+-------+--------+ | TimeMachine | - | 🟢 | +------------------------+-------+--------+ | UserVisibleSettings | - | 🟢 | +------------------------+-------+--------+ Running init script '/quay-registry/conf/init/nginx_conf_create.sh' Running init script '/quay-registry/conf/init/supervisord_conf_create.sh' Running init script '/quay-registry/conf/init/zz_boot.sh' /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? if result is -1: 2023-08-08 05:45:28,780 INFO RPC interface 'supervisor' initialized 2023-08-08 05:45:28,780 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2023-08-08 05:45:28,780 INFO supervisord started with pid 7 2023-08-08 05:45:29,781 INFO spawned: 'stdout' with pid 53 2023-08-08 05:45:29,783 INFO spawned: 'blobuploadcleanupworker' with pid 54 2023-08-08 05:45:29,785 INFO spawned: 'builder' with pid 55 2023-08-08 05:45:29,787 INFO spawned: 'buildlogsarchiver' with pid 56 2023-08-08 05:45:29,788 INFO spawned: 'chunkcleanupworker' with pid 57 2023-08-08 05:45:29,790 INFO spawned: 'dnsmasq' with pid 58 2023-08-08 05:45:29,792 INFO spawned: 'expiredappspecifictokenworker' with pid 59 2023-08-08 05:45:29,794 INFO spawned: 'exportactionlogsworker' with pid 60 2023-08-08 05:45:29,795 INFO spawned: 'gcworker' with pid 61 2023-08-08 05:45:29,797 INFO spawned: 'globalpromstats' with pid 62 2023-08-08 05:45:29,798 INFO spawned: 'gunicorn-registry' with pid 63 2023-08-08 05:45:29,800 INFO spawned: 'gunicorn-secscan' with pid 64 2023-08-08 05:45:29,802 INFO spawned: 'gunicorn-web' with pid 65 2023-08-08 05:45:29,803 INFO spawned: 'logrotateworker' with pid 66 2023-08-08 05:45:29,805 INFO spawned: 'manifestbackfillworker' with pid 67 2023-08-08 05:45:29,806 INFO spawned: 'memcache' with pid 68 2023-08-08 05:45:29,808 INFO spawned: 'namespacegcworker' with pid 69 2023-08-08 05:45:29,809 INFO spawned: 'nginx' with pid 70 2023-08-08 05:45:29,812 INFO spawned: 'notificationworker' with pid 71 2023-08-08 05:45:29,813 INFO spawned: 'pushgateway' with pid 72 2023-08-08 05:45:29,872 INFO spawned: 'queuecleanupworker' with pid 73 2023-08-08 05:45:29,874 INFO spawned: 'repositoryactioncounter' with pid 74 2023-08-08 05:45:29,875 INFO spawned: 'repositorygcworker' with pid 75 2023-08-08 05:45:29,877 INFO spawned: 'securityscanningnotificationworker' with pid 76 2023-08-08 05:45:29,878 INFO spawned: 'securityworker' with pid 77 2023-08-08 05:45:29,880 INFO spawned: 'servicekey' with pid 78 2023-08-08 05:45:29,986 INFO spawned: 'storagereplication' with pid 79 2023-08-08 05:45:29,988 INFO spawned: 'teamsyncworker' with pid 80 2023-08-08 05:45:31,394 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,394 INFO success: blobuploadcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,394 INFO success: builder entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,394 INFO success: buildlogsarchiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,394 INFO success: chunkcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,394 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: expiredappspecifictokenworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: exportactionlogsworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: gcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: globalpromstats entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: gunicorn-registry entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: gunicorn-secscan entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: gunicorn-web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: logrotateworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: manifestbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: memcache entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: namespacegcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: notificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: pushgateway entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: queuecleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: repositoryactioncounter entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: repositorygcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: securityscanningnotificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: securityworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: servicekey entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: storagereplication entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-08-08 05:45:31,395 INFO success: teamsyncworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) dnsmasq stderr | dnsmasq: started, version 2.79 cachesize 150 dnsmasq stderr | dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify dnsmasq stderr | dnsmasq: reading /etc/resolv.conf dnsmasq stderr | dnsmasq: using nameserver 172.30.0.10#53 dnsmasq stderr | dnsmasq: read /etc/hosts - 7 addresses nginx stdout | 2023/08/08 05:45:29 [warn] 70#0: the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:40 nginx stdout | 2023/08/08 05:45:29 [warn] 70#0: the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:67 nginx stdout | 2023/08/08 05:45:29 [warn] 70#0: the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:91 nginx stderr | nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:40 nginx stderr | nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:67 nginx stderr | nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:91 nginx stdout | 2023/08/08 05:45:29 [alert] 90#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2023/08/08 05:45:29 [alert] 92#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2023/08/08 05:45:29 [alert] 94#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2023/08/08 05:45:29 [alert] 89#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2023/08/08 05:45:29 [alert] 91#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2023/08/08 05:45:29 [alert] 96#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2023/08/08 05:45:29 [alert] 95#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2023/08/08 05:45:29 [alert] 93#0: setpriority(-10) failed (13: Permission denied) pushgateway stderr | ts=2023-08-08T05:45:35.870Z caller=main.go:86 level=info msg="starting pushgateway" version="(version=, branch=, revision=unknown)" pushgateway stderr | ts=2023-08-08T05:45:35.870Z caller=main.go:87 level=info build_context="(go=go1.19.10, platform=linux/amd64, user=, date=, tags=unknown)" pushgateway stderr | ts=2023-08-08T05:45:36.293Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9091 pushgateway stderr | ts=2023-08-08T05:45:36.293Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9091 nginx stdout | 2023/08/08 05:45:37 [crit] 90#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.129.2.33:8443" nginx stdout | 2023/08/08 05:45:37 [crit] 90#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.129.2.33:8443" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:45:37 +0000] "GET /health/instance HTTP/2.0" 502 157 "-" "kube-probe/1.26" (0.000 47 0.000 : 0.000) manifestbackfillworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? manifestbackfillworker stderr | if result is -1: storagereplication stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? storagereplication stderr | if result is -1: notificationworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? notificationworker stderr | if result is -1: logrotateworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? logrotateworker stderr | if result is -1: servicekey stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? servicekey stderr | if result is -1: globalpromstats stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? globalpromstats stderr | if result is -1: notificationworker stdout | 2023-08-08 05:45:48,273 [71] [DEBUG] [workers.worker] Scheduling worker. notificationworker stdout | 2023-08-08 05:45:48,274 [71] [INFO] [apscheduler.scheduler] Scheduler started notificationworker stdout | 2023-08-08 05:45:48,292 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:45:48,292 [71] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added notificationworker stdout | 2023-08-08 05:45:48,370 [71] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:45:55.371682 blobuploadcleanupworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? blobuploadcleanupworker stderr | if result is -1: logrotateworker stdout | 2023-08-08 05:45:48,679 [66] [DEBUG] [__main__] Action log rotation worker not enabled; skipping gunicorn-secscan stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? gunicorn-secscan stderr | if result is -1: notificationworker stdout | 2023-08-08 05:45:49,177 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" notificationworker stdout | 2023-08-08 05:45:49,177 [71] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:48:49.178248 notificationworker stdout | 2023-08-08 05:45:49,177 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" notificationworker stdout | 2023-08-08 05:45:49,177 [71] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:02.178530 notificationworker stdout | 2023-08-08 05:45:49,177 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" notificationworker stdout | 2023-08-08 05:45:49,180 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:45:49,180 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:45:55.371682+00:00 (in 6.190909 seconds) repositorygcworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? repositorygcworker stderr | if result is -1: repositoryactioncounter stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? repositoryactioncounter stderr | if result is -1: securityworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? securityworker stderr | if result is -1: builder stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? builder stderr | if result is -1: gcworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? gcworker stderr | if result is -1: manifestbackfillworker stdout | 2023-08-08 05:45:50,197 [67] [DEBUG] [workers.worker] Scheduling worker. manifestbackfillworker stdout | 2023-08-08 05:45:50,198 [67] [INFO] [apscheduler.scheduler] Scheduler started manifestbackfillworker stdout | 2023-08-08 05:45:50,295 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2023-08-08 05:45:50,295 [67] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added manifestbackfillworker stdout | 2023-08-08 05:45:50,370 [67] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 06:19:18.371678 storagereplication stdout | 2023-08-08 05:45:50,596 [79] [DEBUG] [__main__] Full storage replication disabled; skipping manifestbackfillworker stdout | 2023-08-08 05:45:50,583 [67] [INFO] [apscheduler.scheduler] Added job "ManifestBackfillWorker._backfill_manifests" to job store "default" manifestbackfillworker stdout | 2023-08-08 05:45:50,670 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2023-08-08 05:45:50,670 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:18.371678+00:00 (in 2007.700808 seconds) queuecleanupworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? queuecleanupworker stderr | if result is -1: exportactionlogsworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? exportactionlogsworker stderr | if result is -1: chunkcleanupworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? chunkcleanupworker stderr | if result is -1: gunicorn-secscan stdout | 2023-08-08 05:45:52,078 [64] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2023-08-08 05:45:52,078 [64] [INFO] [data.registry_model] Using registry model `` gunicorn-secscan stdout | 2023-08-08 05:45:52,078 [64] [INFO] [data.registry_model] =============================== nginx stdout | 2023/08/08 05:45:52 [crit] 90#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.129.2.33:8443" nginx stdout | 2023/08/08 05:45:52 [crit] 90#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.129.2.33:8443" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:45:52 +0000] "GET /health/instance HTTP/2.0" 502 157 "-" "kube-probe/1.26" (0.000 47 0.000 : 0.000) teamsyncworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? teamsyncworker stderr | if result is -1: gcworker stdout | 2023-08-08 05:45:53,294 [61] [DEBUG] [workers.worker] Scheduling worker. gcworker stdout | 2023-08-08 05:45:53,295 [61] [INFO] [apscheduler.scheduler] Scheduler started gcworker stdout | 2023-08-08 05:45:53,394 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:45:53,394 [61] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added gcworker stdout | 2023-08-08 05:45:53,416 [61] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:45:59.417660 gunicorn-web stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? gunicorn-web stderr | if result is -1: gcworker stdout | 2023-08-08 05:45:53,873 [61] [INFO] [apscheduler.scheduler] Added job "GarbageCollectionWorker._garbage_collection_repos" to job store "default" gcworker stdout | 2023-08-08 05:45:53,878 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:45:53,879 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:45:59.417660+00:00 (in 5.538674 seconds) gunicorn-secscan stdout | 2023-08-08 05:45:54,278 [64] [DEBUG] [app] Loading default config. gunicorn-secscan stdout | 2023-08-08 05:45:54,279 [64] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-secscan stdout | 2023-08-08 05:45:54,579 [64] [DEBUG] [app] Loaded config gunicorn-secscan stdout | 2023-08-08 05:45:54,580 [64] [INFO] [util.ipresolver] Loading AWS IP ranges from disk buildlogsarchiver stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? buildlogsarchiver stderr | if result is -1: chunkcleanupworker stdout | 2023-08-08 05:45:55,189 [57] [DEBUG] [__main__] Starting chunk cleanup worker chunkcleanupworker stdout | 2023-08-08 05:45:55,190 [57] [DEBUG] [workers.worker] Scheduling worker. chunkcleanupworker stdout | 2023-08-08 05:45:55,191 [57] [INFO] [apscheduler.scheduler] Scheduler started chunkcleanupworker stdout | 2023-08-08 05:45:55,281 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:45:55,282 [57] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added chunkcleanupworker stdout | 2023-08-08 05:45:55,292 [57] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:02.293757 blobuploadcleanupworker stdout | 2023-08-08 05:45:55,372 [54] [DEBUG] [workers.worker] Scheduling worker. blobuploadcleanupworker stdout | 2023-08-08 05:45:55,373 [54] [INFO] [apscheduler.scheduler] Scheduler started notificationworker stdout | 2023-08-08 05:45:55,381 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:45:55,392 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:45:55 UTC)" (scheduled at 2023-08-08 05:45:55.371682+00:00) notificationworker stdout | 2023-08-08 05:45:55,392 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:45:55,393 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 45, 55, 392986), True, datetime.datetime(2023, 8, 8, 5, 45, 55, 392986), 0, 'notification/%', 50, 1, 0]) blobuploadcleanupworker stdout | 2023-08-08 05:45:55,394 [54] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2023-08-08 05:45:55,395 [54] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added blobuploadcleanupworker stdout | 2023-08-08 05:45:55,470 [54] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 06:11:41.471663 notificationworker stdout | 2023-08-08 05:45:55,470 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:02.178530+00:00 (in 6.796725 seconds) repositorygcworker stdout | 2023-08-08 05:45:55,698 [75] [DEBUG] [__main__] Starting repository GC worker blobuploadcleanupworker stdout | 2023-08-08 05:45:55,777 [54] [INFO] [apscheduler.scheduler] Added job "BlobUploadCleanupWorker._try_cleanup_uploads" to job store "default" notificationworker stdout | 2023-08-08 05:45:55,704 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:45:55,770 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:45:55,771 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:05 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:45:55,700 [75] [DEBUG] [workers.worker] Scheduling worker. repositorygcworker stdout | 2023-08-08 05:45:55,701 [75] [INFO] [apscheduler.scheduler] Scheduler started gunicorn-secscan stdout | 2023-08-08 05:45:55,781 [64] [DEBUG] [util.ipresolver] Building AWS IP ranges blobuploadcleanupworker stdout | 2023-08-08 05:45:55,781 [54] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2023-08-08 05:45:55,781 [54] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:41.471663+00:00 (in 1545.689843 seconds) repositorygcworker stdout | 2023-08-08 05:45:55,782 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:45:55,782 [75] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositorygcworker stdout | 2023-08-08 05:45:55,792 [75] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:09.793658 chunkcleanupworker stdout | 2023-08-08 05:45:55,873 [57] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" chunkcleanupworker stdout | 2023-08-08 05:45:55,873 [57] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:49:46.874674 chunkcleanupworker stdout | 2023-08-08 05:45:55,873 [57] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" chunkcleanupworker stdout | 2023-08-08 05:45:55,873 [57] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:28.874906 chunkcleanupworker stdout | 2023-08-08 05:45:55,874 [57] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" chunkcleanupworker stdout | 2023-08-08 05:45:55,881 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:45:55,881 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:02.293757+00:00 (in 6.411941 seconds) gunicorn-web stdout | 2023-08-08 05:45:56,171 [65] [INFO] [data.registry_model] =============================== gunicorn-web stdout | 2023-08-08 05:45:56,171 [65] [INFO] [data.registry_model] Using registry model `` gunicorn-web stdout | 2023-08-08 05:45:56,172 [65] [INFO] [data.registry_model] =============================== repositorygcworker stdout | 2023-08-08 05:45:56,186 [75] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" repositorygcworker stdout | 2023-08-08 05:45:56,186 [75] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:49:19.187230 repositorygcworker stdout | 2023-08-08 05:45:56,186 [75] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" repositorygcworker stdout | 2023-08-08 05:45:56,186 [75] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:04.187429 repositorygcworker stdout | 2023-08-08 05:45:56,186 [75] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" repositorygcworker stdout | 2023-08-08 05:45:56,192 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:45:56,192 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:04.187429+00:00 (in 7.994587 seconds) gunicorn-secscan stdout | 2023-08-08 05:45:56,379 [64] [DEBUG] [util.ipresolver] Finished building AWS IP ranges queuecleanupworker stdout | 2023-08-08 05:45:56,574 [73] [DEBUG] [workers.worker] Scheduling worker. queuecleanupworker stdout | 2023-08-08 05:45:56,574 [73] [INFO] [apscheduler.scheduler] Scheduler started queuecleanupworker stdout | 2023-08-08 05:45:56,594 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2023-08-08 05:45:56,594 [73] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added queuecleanupworker stdout | 2023-08-08 05:45:56,595 [73] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 11:26:54.596709 gunicorn-secscan stdout | 2023-08-08 05:45:56,785 [64] [DEBUG] [data.database] Configuring database gunicorn-secscan stdout | 2023-08-08 05:45:56,786 [64] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-secscan stdout | 2023-08-08 05:45:56,787 [64] [INFO] [data.secscan_model] =============================== gunicorn-secscan stdout | 2023-08-08 05:45:56,787 [64] [INFO] [data.secscan_model] Using split secscan model: `[]` gunicorn-secscan stdout | 2023-08-08 05:45:56,787 [64] [INFO] [data.secscan_model] =============================== gunicorn-secscan stdout | 2023-08-08 05:45:56,787 [64] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-secscan stdout | 2023-08-08 05:45:56,787 [64] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2023-08-08 05:45:56,787 [64] [INFO] [data.logs_model] Using logs model `` gunicorn-secscan stdout | 2023-08-08 05:45:56,787 [64] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2023-08-08 05:45:56,889 [64] [DEBUG] [__config__] Starting secscan gunicorn with 2 workers and gevent worker class gunicorn-web stdout | 2023-08-08 05:45:56,883 [65] [DEBUG] [app] Loading default config. gunicorn-web stdout | 2023-08-08 05:45:56,884 [65] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-secscan stderr | Traceback (most recent call last): gunicorn-secscan stderr | File "src/gevent/_abstract_linkable.py", line 287, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-secscan stderr | File "src/gevent/_abstract_linkable.py", line 333, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-secscan stderr | AssertionError: (None, ) gunicorn-secscan stderr | 2023-08-08T05:45:56Z failed with AssertionError gunicorn-web stdout | 2023-08-08 05:45:56,971 [65] [DEBUG] [app] Loaded config gunicorn-web stdout | 2023-08-08 05:45:56,972 [65] [INFO] [util.ipresolver] Loading AWS IP ranges from disk queuecleanupworker stdout | 2023-08-08 05:45:56,993 [73] [INFO] [apscheduler.scheduler] Added job "QueueCleanupWorker._cleanup_queue" to job store "default" queuecleanupworker stdout | 2023-08-08 05:45:57,070 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2023-08-08 05:45:57,071 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 11:26:54.596709+00:00 (in 20457.525704 seconds) namespacegcworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? namespacegcworker stderr | if result is -1: securityscanningnotificationworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? securityscanningnotificationworker stderr | if result is -1: teamsyncworker stdout | 2023-08-08 05:45:57,287 [80] [DEBUG] [__main__] Team syncing is disabled; sleeping gunicorn-web stdout | 2023-08-08 05:45:57,289 [65] [DEBUG] [util.ipresolver] Building AWS IP ranges gunicorn-web stdout | 2023-08-08 05:45:57,675 [65] [DEBUG] [util.ipresolver] Finished building AWS IP ranges repositoryactioncounter stdout | 2023-08-08 05:45:57,687 [74] [DEBUG] [workers.worker] Scheduling worker. repositoryactioncounter stdout | 2023-08-08 05:45:57,687 [74] [INFO] [apscheduler.scheduler] Scheduler started repositoryactioncounter stdout | 2023-08-08 05:45:57,688 [74] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2023-08-08 05:45:57,688 [74] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositoryactioncounter stdout | 2023-08-08 05:45:57,689 [74] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 08:50:48.689974 repositoryactioncounter stdout | 2023-08-08 05:45:57,893 [74] [INFO] [apscheduler.scheduler] Added job "RepositoryActionCountWorker._run_counting" to job store "default" repositoryactioncounter stdout | 2023-08-08 05:45:57,894 [74] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2023-08-08 05:45:57,894 [74] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 08:50:48.689974+00:00 (in 11090.795387 seconds) gunicorn-web stdout | 2023-08-08 05:45:57,990 [65] [DEBUG] [data.database] Configuring database gunicorn-web stdout | 2023-08-08 05:45:57,991 [65] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:45:57,991 [65] [INFO] [data.secscan_model] =============================== gunicorn-web stdout | 2023-08-08 05:45:57,991 [65] [INFO] [data.secscan_model] Using split secscan model: `[]` gunicorn-web stdout | 2023-08-08 05:45:57,992 [65] [INFO] [data.secscan_model] =============================== gunicorn-web stdout | 2023-08-08 05:45:57,992 [65] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-web stdout | 2023-08-08 05:45:57,992 [65] [INFO] [data.logs_model] =============================== gunicorn-web stdout | 2023-08-08 05:45:57,992 [65] [INFO] [data.logs_model] Using logs model `` gunicorn-web stdout | 2023-08-08 05:45:57,992 [65] [INFO] [data.logs_model] =============================== expiredappspecifictokenworker stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? expiredappspecifictokenworker stderr | if result is -1: buildlogsarchiver stdout | 2023-08-08 05:45:58,193 [56] [DEBUG] [workers.worker] Scheduling worker. buildlogsarchiver stdout | 2023-08-08 05:45:58,194 [56] [INFO] [apscheduler.scheduler] Scheduler started buildlogsarchiver stdout | 2023-08-08 05:45:58,195 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:45:58,195 [56] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added buildlogsarchiver stdout | 2023-08-08 05:45:58,196 [56] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:07.197412 buildlogsarchiver stdout | 2023-08-08 05:45:58,471 [56] [INFO] [apscheduler.scheduler] Added job "ArchiveBuildLogsWorker._archive_redis_buildlogs" to job store "default" buildlogsarchiver stdout | 2023-08-08 05:45:58,472 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:45:58,472 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:07.197412+00:00 (in 8.724864 seconds) builder stdout | 2023-08-08 05:45:58,774 [55] [DEBUG] [__main__] Asking to start build manager with lifecycle "ephemeral" builder stdout | 2023-08-08 05:45:58,774 [55] [DEBUG] [__main__] Will pass buildman hostname quayregistry-quay-builder-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com:443 to builders for websocket connection builder stdout | 2023-08-08 05:45:58,774 [55] [DEBUG] [__main__] Starting build manager with lifecycle "ephemeral" builder stdout | 2023-08-08 05:45:58,774 [55] [DEBUG] [buildman.server] Initializing the lifecycle manager builder stdout | 2023-08-08 05:45:58,775 [55] [DEBUG] [buildman.manager.ephemeral] calling orchestrator_from_config builder stdout | 2023-08-08 05:45:58,775 [55] [DEBUG] [buildman.orchestrator] attempting to create orchestrator redis with kwargs {'host': 'quayregistry-quay-redis', 'password': '', 'skip_keyspace_event_setup': False, 'ssl': False, 'orchestrator_prefix': 'buildman/production/', 'canceller_only': False} builder stdout | 2023-08-08 05:45:58,775 [55] [DEBUG] [buildman.orchestrator] creating pubsub with key buildman/production/orchestrator_events builder stdout | 2023-08-08 05:45:58,779 [55] [DEBUG] [buildman.manager.ephemeral] setting on_key_change callbacks for job expiry, cancel builder stdout | 2023-08-08 05:45:58,779 [55] [DEBUG] [buildman.orchestrator] watching key: buildman/production/building/ builder stdout | 2023-08-08 05:45:58,779 [55] [DEBUG] [buildman.orchestrator] watching key: buildman/production/cancel/ builder stdout | 2023-08-08 05:45:58,779 [55] [DEBUG] [buildman.server] Initializing the gRPC server builder stdout | 2023-08-08 05:45:58,782 [55] [DEBUG] [buildman.server] Starting the gRPC server... builder stdout | 2023-08-08 05:45:58,794 [55] [DEBUG] [buildman.server] Starting the build manager... builder stdout | 2023-08-08 05:45:58,794 [55] [DEBUG] [buildman.manager.ephemeral] Initializing work checker builder stdout | 2023-08-08 05:45:58,794 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:45:59,292 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:45:59,470 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:45:59,486 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:45:59 UTC)" (scheduled at 2023-08-08 05:45:59.417660+00:00) gcworker stdout | 2023-08-08 05:45:59,487 [61] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2023-08-08 05:45:59,488 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:09.417660+00:00 (in 9.946786 seconds) gcworker stdout | 2023-08-08 05:45:59,684 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690263959682, 1209600, True, 3, 500, 1, 0]) securityworker stdout | 2023-08-08 05:45:59,685 [77] [DEBUG] [workers.worker] Scheduling worker. securityworker stdout | 2023-08-08 05:45:59,685 [77] [INFO] [apscheduler.scheduler] Scheduler started securityworker stdout | 2023-08-08 05:45:59,686 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:45:59,686 [77] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityworker stdout | 2023-08-08 05:45:59,686 [77] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:19.687363 gcworker stdout | 2023-08-08 05:45:59,788 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:45:59,788 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:45:59,788 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:09 UTC)" executed successfully securityworker stdout | 2023-08-08 05:45:59,889 [77] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_in_scanner" to job store "default" securityworker stdout | 2023-08-08 05:45:59,889 [77] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:26.890741 securityworker stdout | 2023-08-08 05:45:59,889 [77] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_recent_manifests_in_scanner" to job store "default" securityworker stdout | 2023-08-08 05:45:59,890 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:45:59,890 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:19.687363+00:00 (in 19.797302 seconds) servicekey stdout | 2023-08-08 05:46:00,297 [78] [DEBUG] [workers.worker] Scheduling worker. servicekey stdout | 2023-08-08 05:46:00,299 [78] [INFO] [apscheduler.scheduler] Scheduler started servicekey stdout | 2023-08-08 05:46:00,372 [78] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2023-08-08 05:46:00,372 [78] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added servicekey stdout | 2023-08-08 05:46:00,372 [78] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 06:00:10.373311 servicekey stdout | 2023-08-08 05:46:00,670 [78] [INFO] [apscheduler.scheduler] Added job "ServiceKeyWorker._refresh_service_key" to job store "default" servicekey stdout | 2023-08-08 05:46:00,671 [78] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2023-08-08 05:46:00,671 [78] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:10.373311+00:00 (in 849.701746 seconds) globalpromstats stdout | 2023-08-08 05:46:00,692 [62] [DEBUG] [workers.worker] Scheduling worker. globalpromstats stdout | 2023-08-08 05:46:00,692 [62] [INFO] [apscheduler.scheduler] Scheduler started globalpromstats stdout | 2023-08-08 05:46:00,692 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2023-08-08 05:46:00,692 [62] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added globalpromstats stdout | 2023-08-08 05:46:00,692 [62] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 06:23:09.693962 globalpromstats stdout | 2023-08-08 05:46:00,886 [62] [INFO] [apscheduler.scheduler] Added job "GlobalPrometheusStatsWorker._try_report_stats" to job store "default" globalpromstats stdout | 2023-08-08 05:46:00,886 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2023-08-08 05:46:00,886 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:09.693962+00:00 (in 2228.807236 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:46:01,093 [76] [DEBUG] [__main__] Starting security scanning notification worker securityscanningnotificationworker stdout | 2023-08-08 05:46:01,095 [76] [DEBUG] [workers.worker] Scheduling worker. securityscanningnotificationworker stdout | 2023-08-08 05:46:01,095 [76] [INFO] [apscheduler.scheduler] Scheduler started securityscanningnotificationworker stdout | 2023-08-08 05:46:01,096 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:46:01,096 [76] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:13.097375 securityscanningnotificationworker stdout | 2023-08-08 05:46:01,096 [76] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityscanningnotificationworker stdout | 2023-08-08 05:46:01,282 [76] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" securityscanningnotificationworker stdout | 2023-08-08 05:46:01,282 [76] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:31.283850 securityscanningnotificationworker stdout | 2023-08-08 05:46:01,283 [76] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" securityscanningnotificationworker stdout | 2023-08-08 05:46:01,283 [76] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:54.284093 securityscanningnotificationworker stdout | 2023-08-08 05:46:01,283 [76] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" securityscanningnotificationworker stdout | 2023-08-08 05:46:01,283 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:46:01,283 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:13.097375+00:00 (in 11.813860 seconds) namespacegcworker stdout | 2023-08-08 05:46:01,390 [69] [DEBUG] [__main__] Starting namespace GC worker namespacegcworker stdout | 2023-08-08 05:46:01,391 [69] [DEBUG] [workers.worker] Scheduling worker. namespacegcworker stdout | 2023-08-08 05:46:01,392 [69] [INFO] [apscheduler.scheduler] Scheduler started namespacegcworker stdout | 2023-08-08 05:46:01,392 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:46:01,392 [69] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added namespacegcworker stdout | 2023-08-08 05:46:01,392 [69] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:54.393627 exportactionlogsworker stdout | 2023-08-08 05:46:01,394 [60] [DEBUG] [__main__] Starting export action logs worker exportactionlogsworker stdout | 2023-08-08 05:46:01,395 [60] [DEBUG] [workers.worker] Scheduling worker. exportactionlogsworker stdout | 2023-08-08 05:46:01,396 [60] [INFO] [apscheduler.scheduler] Scheduler started exportactionlogsworker stdout | 2023-08-08 05:46:01,396 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:46:01,397 [60] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added exportactionlogsworker stdout | 2023-08-08 05:46:01,397 [60] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:47.398551 expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,511 [59] [DEBUG] [__main__] Starting expired app specific token GC worker expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,511 [59] [DEBUG] [__main__] Found expiration window: 1d expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,512 [59] [DEBUG] [workers.worker] Scheduling worker. expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,513 [59] [INFO] [apscheduler.scheduler] Scheduler started expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,570 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:46:01,512 [60] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" exportactionlogsworker stdout | 2023-08-08 05:46:01,513 [60] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:48:33.514094 exportactionlogsworker stdout | 2023-08-08 05:46:01,513 [60] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" exportactionlogsworker stdout | 2023-08-08 05:46:01,513 [60] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:25.514341 exportactionlogsworker stdout | 2023-08-08 05:46:01,513 [60] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" exportactionlogsworker stdout | 2023-08-08 05:46:01,513 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:46:01,513 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:25.514341+00:00 (in 24.000586 seconds) expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,571 [59] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,571 [59] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 06:07:03.572062 namespacegcworker stdout | 2023-08-08 05:46:01,578 [69] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" namespacegcworker stdout | 2023-08-08 05:46:01,578 [69] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:49:39.579576 namespacegcworker stdout | 2023-08-08 05:46:01,578 [69] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" namespacegcworker stdout | 2023-08-08 05:46:01,578 [69] [DEBUG] [workers.worker] First run scheduled for 2023-08-08 05:46:36.579841 namespacegcworker stdout | 2023-08-08 05:46:01,578 [69] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" namespacegcworker stdout | 2023-08-08 05:46:01,579 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:46:01,579 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:36.579841+00:00 (in 35.000641 seconds) gunicorn-web stdout | 2023-08-08 05:46:01,610 [65] [DEBUG] [__config__] Starting web gunicorn with 4 workers and gevent worker class gunicorn-web stderr | Traceback (most recent call last): gunicorn-web stderr | File "src/gevent/_abstract_linkable.py", line 287, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-web stderr | File "src/gevent/_abstract_linkable.py", line 333, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-web stderr | AssertionError: (None, ) gunicorn-web stderr | 2023-08-08T05:46:01Z failed with AssertionError expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,624 [59] [INFO] [apscheduler.scheduler] Added job "ExpiredAppSpecificTokenWorker._gc_expired_tokens" to job store "default" expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,624 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2023-08-08 05:46:01,625 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:03.572062+00:00 (in 1261.947034 seconds) gunicorn-registry stderr | /app/lib/python3.9/site-packages/magic.py:193: SyntaxWarning: "is" with a literal. Did you mean "=="? gunicorn-registry stderr | if result is -1: notificationworker stdout | 2023-08-08 05:46:02,267 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:46:02,268 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:05.371682+00:00 (in 3.103739 seconds) notificationworker stdout | 2023-08-08 05:46:02,268 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:02 UTC)" (scheduled at 2023-08-08 05:46:02.178530+00:00) notificationworker stdout | 2023-08-08 05:46:02,268 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:46:02,268 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:46:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:46:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:02 UTC)" (scheduled at 2023-08-08 05:46:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:46:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:46:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 2, 294705), True, datetime.datetime(2023, 8, 8, 5, 46, 2, 294705), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:46:02,296 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:12.293757+00:00 (in 9.999626 seconds) chunkcleanupworker stdout | 2023-08-08 05:46:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:46:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:46:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:46:02,559 [63] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2023-08-08 05:46:02,559 [63] [INFO] [data.registry_model] Using registry model `` gunicorn-registry stdout | 2023-08-08 05:46:02,559 [63] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2023-08-08 05:46:02,735 [63] [DEBUG] [app] Loading default config. gunicorn-registry stdout | 2023-08-08 05:46:02,736 [63] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-registry stdout | 2023-08-08 05:46:02,750 [63] [DEBUG] [app] Loaded config gunicorn-registry stdout | 2023-08-08 05:46:02,751 [63] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-registry stdout | 2023-08-08 05:46:02,759 [63] [DEBUG] [util.ipresolver] Building AWS IP ranges gunicorn-registry stdout | 2023-08-08 05:46:02,843 [63] [DEBUG] [util.ipresolver] Finished building AWS IP ranges gunicorn-registry stdout | 2023-08-08 05:46:02,872 [63] [DEBUG] [data.database] Configuring database gunicorn-registry stdout | 2023-08-08 05:46:02,873 [63] [INFO] [data.database] Connection pooling enabled for postgresql; stale timeout: None; max connection count: None gunicorn-registry stdout | 2023-08-08 05:46:02,874 [63] [INFO] [data.secscan_model] =============================== gunicorn-registry stdout | 2023-08-08 05:46:02,874 [63] [INFO] [data.secscan_model] Using split secscan model: `[]` gunicorn-registry stdout | 2023-08-08 05:46:02,874 [63] [INFO] [data.secscan_model] =============================== gunicorn-registry stdout | 2023-08-08 05:46:02,874 [63] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-registry stdout | 2023-08-08 05:46:02,874 [63] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2023-08-08 05:46:02,874 [63] [INFO] [data.logs_model] Using logs model `` gunicorn-registry stdout | 2023-08-08 05:46:02,958 [63] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2023-08-08 05:46:03,098 [63] [DEBUG] [__config__] Starting registry gunicorn with 8 workers and gevent worker class gunicorn-registry stderr | Traceback (most recent call last): gunicorn-registry stderr | File "src/gevent/_abstract_linkable.py", line 287, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-registry stderr | File "src/gevent/_abstract_linkable.py", line 333, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-registry stderr | AssertionError: (None, ) gunicorn-registry stderr | 2023-08-08T05:46:03Z failed with AssertionError repositorygcworker stdout | 2023-08-08 05:46:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:46:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:46:04 UTC)" (scheduled at 2023-08-08 05:46:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:46:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:46:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:46:04 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:46:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:09.793658+00:00 (in 5.605837 seconds) notificationworker stdout | 2023-08-08 05:46:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:46:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:15.371682+00:00 (in 9.999493 seconds) notificationworker stdout | 2023-08-08 05:46:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:15 UTC)" (scheduled at 2023-08-08 05:46:05.371682+00:00) notificationworker stdout | 2023-08-08 05:46:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:46:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 5, 372881), True, datetime.datetime(2023, 8, 8, 5, 46, 5, 372881), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:46:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:46:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:46:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:46:07,182 [197] [DEBUG] [app] Starting request: urn:request:7f7942ba-bd05-489b-8716-df4156aae7f0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:46:07,190 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 buildlogsarchiver stdout | 2023-08-08 05:46:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:46:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:46:07 UTC)" (scheduled at 2023-08-08 05:46:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:46:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 46, 7, 198574), False, 50, 1, 0]) buildlogsarchiver stdout | 2023-08-08 05:46:07,199 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:37.197412+00:00 (in 29.999645 seconds) buildlogsarchiver stdout | 2023-08-08 05:46:07,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:46:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:46:07,207 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:46:37 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:46:07,223 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:46:07,227 [209] [DEBUG] [app] Starting request: urn:request:a947c1b6-816c-4f94-a7cf-4d28389c1834 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:46:07,229 [209] [DEBUG] [app] Ending request: urn:request:a947c1b6-816c-4f94-a7cf-4d28389c1834 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a947c1b6-816c-4f94-a7cf-4d28389c1834', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.006 162 0.006) gunicorn-registry stdout | 2023-08-08 05:46:07,229 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:07,230 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:07,232 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:07,235 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:07,237 [197] [DEBUG] [app] Starting request: urn:request:65411570-da69-41f9-bccf-a8a66d132ee2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:07,238 [197] [DEBUG] [app] Ending request: urn:request:65411570-da69-41f9-bccf-a8a66d132ee2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:65411570-da69-41f9-bccf-a8a66d132ee2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2023-08-08 05:46:07,239 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:07,239 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:07,242 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2023, 8, 8, 5, 46, 7, 240682), None, 'quay', 'quay', datetime.datetime(2023, 8, 8, 5, 46, 7, 240714), 'quay', None, datetime.datetime(2023, 8, 7, 5, 46, 7, 240740), datetime.datetime(2023, 8, 1, 5, 46, 7, 240753), None]) gunicorn-web stdout | 2023-08-08 05:46:07,251 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:46:07,255 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:46:07,257 [197] [DEBUG] [app] Ending request: urn:request:7f7942ba-bd05-489b-8716-df4156aae7f0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7f7942ba-bd05-489b-8716-df4156aae7f0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:46:07,257 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:46:07,257 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:46:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:46:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.080 47 0.079) gunicorn-web stdout | 2023-08-08 05:46:07,261 [197] [DEBUG] [app] Starting request: urn:request:f2877c8a-2865-4fb1-8414-7ce9dbdbe5c4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:46:07,263 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:46:07,267 [206] [DEBUG] [app] Starting request: urn:request:5a28cfe4-bda5-45c7-8acc-2e211d411daf (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:46:07,269 [206] [DEBUG] [app] Ending request: urn:request:5a28cfe4-bda5-45c7-8acc-2e211d411daf (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5a28cfe4-bda5-45c7-8acc-2e211d411daf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:46:07,270 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:07,270 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.006 162 0.006) gunicorn-web stdout | 2023-08-08 05:46:07,272 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:07,275 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:07,281 [200] [DEBUG] [app] Starting request: urn:request:1299a228-db27-4934-acfe-777e3223710c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:07,283 [200] [DEBUG] [app] Ending request: urn:request:1299a228-db27-4934-acfe-777e3223710c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1299a228-db27-4934-acfe-777e3223710c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.009 159 0.008) gunicorn-web stdout | 2023-08-08 05:46:07,284 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:07,285 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:07,286 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:46:07,290 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:46:07,296 [197] [DEBUG] [app] Ending request: urn:request:f2877c8a-2865-4fb1-8414-7ce9dbdbe5c4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f2877c8a-2865-4fb1-8414-7ce9dbdbe5c4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:46:07,296 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:46:07,296 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:46:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:46:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.036 47 0.035) builder stdout | 2023-08-08 05:46:09,300 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:46:09,309 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:46:09,309 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:46:09,314 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:09,435 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:46:09,435 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:19.417660+00:00 (in 9.982467 seconds) gcworker stdout | 2023-08-08 05:46:09,435 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:19 UTC)" (scheduled at 2023-08-08 05:46:09.417660+00:00) gcworker stdout | 2023-08-08 05:46:09,436 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690263969435, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:46:09,444 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:46:09,444 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:09,444 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:46:09,795 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:46:09,795 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:04.187429+00:00 (in 54.392281 seconds) repositorygcworker stdout | 2023-08-08 05:46:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:09 UTC)" (scheduled at 2023-08-08 05:46:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:46:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:46:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 9, 795537), True, datetime.datetime(2023, 8, 8, 5, 46, 9, 795537), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:46:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:46:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:46:09,804 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:46:12,295 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:46:12,296 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:22.293757+00:00 (in 9.997740 seconds) chunkcleanupworker stdout | 2023-08-08 05:46:12,296 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:22 UTC)" (scheduled at 2023-08-08 05:46:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:46:12,296 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:46:12,297 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 12, 296385), True, datetime.datetime(2023, 8, 8, 5, 46, 12, 296385), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:46:12,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:46:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:46:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:46:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:46:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:46:13 UTC)" (scheduled at 2023-08-08 05:46:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:46:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:46:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 13, 98436), True, datetime.datetime(2023, 8, 8, 5, 46, 13, 98436), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:46:13,100 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:31.283850+00:00 (in 18.186065 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:46:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:46:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:46:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:46:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:46:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:25.371682+00:00 (in 9.999449 seconds) notificationworker stdout | 2023-08-08 05:46:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:25 UTC)" (scheduled at 2023-08-08 05:46:15.371682+00:00) notificationworker stdout | 2023-08-08 05:46:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:46:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 15, 372784), True, datetime.datetime(2023, 8, 8, 5, 46, 15, 372784), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:46:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:46:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:46:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:46:16,696 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:46:17,906 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:46:19,103 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} builder stdout | 2023-08-08 05:46:19,324 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:46:19,333 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:46:19,333 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:46:19,337 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:46:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:29.417660+00:00 (in 9.999495 seconds) gcworker stdout | 2023-08-08 05:46:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:29 UTC)" (scheduled at 2023-08-08 05:46:19.417660+00:00) gcworker stdout | 2023-08-08 05:46:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690263979418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:46:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:46:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:29 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:46:19,527 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityworker stdout | 2023-08-08 05:46:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:46:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:46:19 UTC)" (scheduled at 2023-08-08 05:46:19.687363+00:00) securityworker stdout | 2023-08-08 05:46:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:46:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:46:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:26.890741+00:00 (in 7.203015 seconds) securityworker stdout | 2023-08-08 05:46:19,691 [77] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 securityworker stdout | 2023-08-08 05:46:19,711 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:46:19,713 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:46:19,719 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:46:19,720 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:46:19,720 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:46:49 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:46:22,179 [197] [DEBUG] [app] Starting request: urn:request:5557081c-7af8-4691-960f-951c0a16c6ac (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:46:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:22,182 [198] [DEBUG] [app] Starting request: urn:request:ae4aacc8-c082-4cfb-b006-5a119cba3a25 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:46:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:46:22,188 [208] [DEBUG] [app] Starting request: urn:request:64d6d18b-2f13-4a2a-9a9a-388a8be731e0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:22,189 [198] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-registry stdout | 2023-08-08 05:46:22,190 [208] [DEBUG] [app] Ending request: urn:request:64d6d18b-2f13-4a2a-9a9a-388a8be731e0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:64d6d18b-2f13-4a2a-9a9a-388a8be731e0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.006 162 0.006) gunicorn-registry stdout | 2023-08-08 05:46:22,191 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:22,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:22,192 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:22,196 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:22,197 [197] [DEBUG] [app] Starting request: urn:request:7d57ff77-dd47-4247-a324-c9bf1a4d7bb7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:22,198 [197] [DEBUG] [app] Ending request: urn:request:7d57ff77-dd47-4247-a324-c9bf1a4d7bb7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7d57ff77-dd47-4247-a324-c9bf1a4d7bb7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:46:22,198 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:22,198 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:22,199 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:46:22,204 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:46:22,210 [197] [DEBUG] [app] Ending request: urn:request:5557081c-7af8-4691-960f-951c0a16c6ac (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5557081c-7af8-4691-960f-951c0a16c6ac', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:46:22,210 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:46:22,210 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:46:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:46:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.032 47 0.033) gunicorn-web stdout | 2023-08-08 05:46:22,222 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:46:22,223 [208] [DEBUG] [app] Starting request: urn:request:95f0edcc-aef1-4c46-a040-484edf372fe4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:46:22,224 [208] [DEBUG] [app] Ending request: urn:request:95f0edcc-aef1-4c46-a040-484edf372fe4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:95f0edcc-aef1-4c46-a040-484edf372fe4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:46:22,224 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:22,225 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:22,227 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:22,229 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:22,230 [197] [DEBUG] [app] Starting request: urn:request:89e8f182-8f16-4d4a-9b41-f6beb17fc7b4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:22,231 [197] [DEBUG] [app] Ending request: urn:request:89e8f182-8f16-4d4a-9b41-f6beb17fc7b4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:89e8f182-8f16-4d4a-9b41-f6beb17fc7b4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:46:22,231 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:22,231 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:46:22,234 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2023, 8, 8, 5, 46, 22, 233007), None, 'quay', 'quay', datetime.datetime(2023, 8, 8, 5, 46, 22, 233035), 'quay', None, datetime.datetime(2023, 8, 7, 5, 46, 22, 233061), datetime.datetime(2023, 8, 1, 5, 46, 22, 233071), None]) gunicorn-web stdout | 2023-08-08 05:46:22,243 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:46:22,248 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:46:22,250 [198] [DEBUG] [app] Ending request: urn:request:ae4aacc8-c082-4cfb-b006-5a119cba3a25 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ae4aacc8-c082-4cfb-b006-5a119cba3a25', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:46:22,250 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:46:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.072 47 0.073) gunicorn-web stdout | 2023-08-08 05:46:22,251 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:46:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:46:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:46:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:28.874906+00:00 (in 6.580591 seconds) chunkcleanupworker stdout | 2023-08-08 05:46:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:32 UTC)" (scheduled at 2023-08-08 05:46:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:46:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:46:22,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 22, 294819), True, datetime.datetime(2023, 8, 8, 5, 46, 22, 294819), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:46:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:46:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:46:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:46:22,328 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:46:24,077 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:46:24,209 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:46:24,488 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:46:24,603 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:46:24,605 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:46:24,607 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:46:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:46:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:35.371682+00:00 (in 9.999468 seconds) notificationworker stdout | 2023-08-08 05:46:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:35 UTC)" (scheduled at 2023-08-08 05:46:25.371682+00:00) notificationworker stdout | 2023-08-08 05:46:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:46:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 25, 372740), True, datetime.datetime(2023, 8, 8, 5, 46, 25, 372740), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:46:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:46:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:46:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:46:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:46:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:46:25 UTC)" (scheduled at 2023-08-08 05:46:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:46:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:46:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:46:25 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:46:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:47.398551+00:00 (in 21.883800 seconds) teamsyncworker stdout | 2023-08-08 05:46:25,527 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:46:26,077 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:46:26,290 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:46:26,892 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:46:26,892 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:49.687363+00:00 (in 22.795129 seconds) securityworker stdout | 2023-08-08 05:46:26,893 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:46:56 UTC)" (scheduled at 2023-08-08 05:46:26.890741+00:00) securityworker stdout | 2023-08-08 05:46:26,893 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:46:26,893 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:46:26,894 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:46:26,894 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:46:26,894 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:46:26,894 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:46:26,894 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:46:26,897 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:46:26,898 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:46:26,904 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:46:26,904 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:46:26,904 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:46:26,904 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:46:26,904 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:46:26,905 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:46:26,905 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:46:26,905 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:46:26,905 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:46:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:46:26,980 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:46:26,981 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:46:26,994 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:46:26,996 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:46:26,997 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:46:27,512 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:46:27,629 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:46:28,710 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 05:46:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:46:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:32.293757+00:00 (in 3.418225 seconds) chunkcleanupworker stdout | 2023-08-08 05:46:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:28 UTC)" (scheduled at 2023-08-08 05:46:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:46:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:46:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:28 UTC)" executed successfully builder stdout | 2023-08-08 05:46:29,347 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:46:29,364 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:46:29,364 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:46:29,369 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:46:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:39.417660+00:00 (in 9.999482 seconds) gcworker stdout | 2023-08-08 05:46:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:39 UTC)" (scheduled at 2023-08-08 05:46:29.417660+00:00) gcworker stdout | 2023-08-08 05:46:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690263989418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:46:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:46:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:46:29,511 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:46:29,600 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:46:30,123 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:46:30,706 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:46:30,978 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:46:31,205 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} securityscanningnotificationworker stdout | 2023-08-08 05:46:31,286 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:46:31,286 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:54.284093+00:00 (in 22.997425 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:46:31,287 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:51:31 UTC)" (scheduled at 2023-08-08 05:46:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:46:31,287 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 46, 31, 287184), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:46:31,294 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 46, 31, 287184), True, datetime.datetime(2023, 8, 8, 5, 46, 31, 287184), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:46:31,296 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 46, 31, 287184), True, datetime.datetime(2023, 8, 8, 5, 46, 31, 287184), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 5, 46, 31, 287184), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:46:31,297 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:46:31,298 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:51:31 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:46:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:46:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:42.293757+00:00 (in 9.999416 seconds) chunkcleanupworker stdout | 2023-08-08 05:46:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:42 UTC)" (scheduled at 2023-08-08 05:46:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:46:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:46:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 32, 294895), True, datetime.datetime(2023, 8, 8, 5, 46, 32, 294895), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:46:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:46:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:46:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:46:32,782 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:46:32,784 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:46:32,787 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:46:32,789 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:46:32,791 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:46:32,794 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:46:32,796 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:46:32,801 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 05:46:32,803 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} notificationworker stdout | 2023-08-08 05:46:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:46:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:45.371682+00:00 (in 9.999469 seconds) notificationworker stdout | 2023-08-08 05:46:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:45 UTC)" (scheduled at 2023-08-08 05:46:35.371682+00:00) notificationworker stdout | 2023-08-08 05:46:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:46:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 35, 372696), True, datetime.datetime(2023, 8, 8, 5, 46, 35, 372696), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:46:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:46:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:46:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:46:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:46:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:46:36 UTC)" (scheduled at 2023-08-08 05:46:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:46:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:46:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:46:36 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:46:36,581 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:54.393627+00:00 (in 17.813422 seconds) gunicorn-web stdout | 2023-08-08 05:46:37,179 [197] [DEBUG] [app] Starting request: urn:request:3fd1a996-38c3-4649-9241-922ce5433f72 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:46:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:37,181 [199] [DEBUG] [app] Starting request: urn:request:2aa003a9-0dab-4725-8a0c-d43179030d06 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:46:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:46:37,185 [208] [DEBUG] [app] Starting request: urn:request:4364a1f5-143e-4010-8f6d-204c26e52c3b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:46:37,186 [208] [DEBUG] [app] Ending request: urn:request:4364a1f5-143e-4010-8f6d-204c26e52c3b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4364a1f5-143e-4010-8f6d-204c26e52c3b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:46:37,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:46:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:37,187 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2023-08-08 05:46:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:37,191 [197] [DEBUG] [app] Starting request: urn:request:4c584753-221e-4ab3-ab05-b8e9996e3bdd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:37,192 [197] [DEBUG] [app] Ending request: urn:request:4c584753-221e-4ab3-ab05-b8e9996e3bdd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4c584753-221e-4ab3-ab05-b8e9996e3bdd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:46:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:46:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:46:37,199 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:46:37,199 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:07.197412+00:00 (in 29.997661 seconds) buildlogsarchiver stdout | 2023-08-08 05:46:37,200 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:07 UTC)" (scheduled at 2023-08-08 05:46:37.197412+00:00) gunicorn-web stdout | 2023-08-08 05:46:37,200 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:46:37,201 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 46, 37, 200370), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:46:37,204 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:37,207 [197] [DEBUG] [app] Ending request: urn:request:3fd1a996-38c3-4649-9241-922ce5433f72 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3fd1a996-38c3-4649-9241-922ce5433f72', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:46:37,207 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 05:46:37,207 [204] [DEBUG] [app] Starting request: urn:request:ea45adff-3970-4ca7-89aa-556e92b5abf0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:37,207 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:46:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:46:37,208 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:46:37,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:46:37,208 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:07 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:46:37,209 [204] [DEBUG] [app] Ending request: urn:request:ea45adff-3970-4ca7-89aa-556e92b5abf0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ea45adff-3970-4ca7-89aa-556e92b5abf0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.005 162 0.005) gunicorn-registry stdout | 2023-08-08 05:46:37,209 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:37,210 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:46:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.032 47 0.031) gunicorn-web stdout | 2023-08-08 05:46:37,212 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:37,214 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:37,216 [200] [DEBUG] [app] Starting request: urn:request:5900dad1-6a9a-4e3e-a959-bda84c9c1660 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:37,216 [200] [DEBUG] [app] Ending request: urn:request:5900dad1-6a9a-4e3e-a959-bda84c9c1660 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5900dad1-6a9a-4e3e-a959-bda84c9c1660', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:46:37,217 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:37,217 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:37,219 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2023, 8, 8, 5, 46, 37, 218431), None, 'quay', 'quay', datetime.datetime(2023, 8, 8, 5, 46, 37, 218464), 'quay', None, datetime.datetime(2023, 8, 7, 5, 46, 37, 218498), datetime.datetime(2023, 8, 1, 5, 46, 37, 218509), None]) gunicorn-web stdout | 2023-08-08 05:46:37,228 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:46:37,232 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:46:37,234 [199] [DEBUG] [app] Ending request: urn:request:2aa003a9-0dab-4725-8a0c-d43179030d06 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2aa003a9-0dab-4725-8a0c-d43179030d06', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:46:37,234 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:46:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.057 47 0.057) gunicorn-web stdout | 2023-08-08 05:46:37,235 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:46:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" builder stdout | 2023-08-08 05:46:39,379 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:46:39,388 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:46:39,388 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:46:39,393 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:46:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:49.417660+00:00 (in 9.999487 seconds) gcworker stdout | 2023-08-08 05:46:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:49 UTC)" (scheduled at 2023-08-08 05:46:39.417660+00:00) gcworker stdout | 2023-08-08 05:46:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690263999418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:46:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:46:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:46:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:46:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:52.293757+00:00 (in 9.999392 seconds) chunkcleanupworker stdout | 2023-08-08 05:46:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:52 UTC)" (scheduled at 2023-08-08 05:46:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:46:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:46:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 42, 294921), True, datetime.datetime(2023, 8, 8, 5, 46, 42, 294921), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:46:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:46:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:46:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:46:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:46:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:55.371682+00:00 (in 9.999460 seconds) notificationworker stdout | 2023-08-08 05:46:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:55 UTC)" (scheduled at 2023-08-08 05:46:45.371682+00:00) notificationworker stdout | 2023-08-08 05:46:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:46:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 45, 373389), True, datetime.datetime(2023, 8, 8, 5, 46, 45, 373389), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:46:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:46:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:46:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:46:46,705 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:46:47,399 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:46:47,400 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:25.514341+00:00 (in 38.114243 seconds) exportactionlogsworker stdout | 2023-08-08 05:46:47,400 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:47 UTC)" (scheduled at 2023-08-08 05:46:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:46:47,400 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:46:47,402 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 47, 400817), True, datetime.datetime(2023, 8, 8, 5, 46, 47, 400817), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:46:47,409 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:46:47,409 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:46:47,409 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:46:47,917 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:46:49,110 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} builder stdout | 2023-08-08 05:46:49,400 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:46:49,409 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:46:49,409 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:46:49,413 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:46:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:59.417660+00:00 (in 9.999463 seconds) gcworker stdout | 2023-08-08 05:46:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:59 UTC)" (scheduled at 2023-08-08 05:46:49.417660+00:00) gcworker stdout | 2023-08-08 05:46:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264009418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:46:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:46:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:46:59 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:46:49,561 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityworker stdout | 2023-08-08 05:46:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:46:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:46:56.890741+00:00 (in 7.202747 seconds) securityworker stdout | 2023-08-08 05:46:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:19 UTC)" (scheduled at 2023-08-08 05:46:49.687363+00:00) securityworker stdout | 2023-08-08 05:46:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:46:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:46:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:46:49,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:46:49,700 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:46:49,701 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:46:49,701 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:19 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:46:52,178 [199] [DEBUG] [app] Starting request: urn:request:341eafdf-a26b-4b45-bf55-498728fd5847 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:46:52,179 [200] [DEBUG] [app] Starting request: urn:request:7bc1949a-ff85-439d-ae6e-80d46b5f3991 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:46:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:52,184 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-registry stdout | 2023-08-08 05:46:52,186 [207] [DEBUG] [app] Starting request: urn:request:dc7b1a31-6e08-4bd1-834c-84a9ad8a1858 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:46:52,188 [207] [DEBUG] [app] Ending request: urn:request:dc7b1a31-6e08-4bd1-834c-84a9ad8a1858 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:dc7b1a31-6e08-4bd1-834c-84a9ad8a1858', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.004 162 0.005) gunicorn-registry stdout | 2023-08-08 05:46:52,188 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:52,188 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:52,190 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:52,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:52,193 [198] [DEBUG] [app] Starting request: urn:request:a754aaa8-fa15-4c25-9603-e9f1550acc98 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:52,194 [198] [DEBUG] [app] Ending request: urn:request:a754aaa8-fa15-4c25-9603-e9f1550acc98 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a754aaa8-fa15-4c25-9603-e9f1550acc98', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:46:52,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:52,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:52,195 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:46:52,200 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:46:52,201 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:46:52,202 [207] [DEBUG] [app] Starting request: urn:request:30f66d90-b10b-4768-832e-6c53c1ee7c3e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:46:52,202 [207] [DEBUG] [app] Ending request: urn:request:30f66d90-b10b-4768-832e-6c53c1ee7c3e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:30f66d90-b10b-4768-832e-6c53c1ee7c3e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 05:46:52,202 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:46:52,203 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:52,204 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:46:52,207 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:46:52,208 [198] [DEBUG] [app] Starting request: urn:request:f55ae1a6-1315-4e9a-ba38-5b5f3cff6d77 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:46:52,208 [198] [DEBUG] [app] Ending request: urn:request:f55ae1a6-1315-4e9a-ba38-5b5f3cff6d77 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f55ae1a6-1315-4e9a-ba38-5b5f3cff6d77', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:46:52,208 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:46:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:46:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:46:52,209 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:46:52,211 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2023, 8, 8, 5, 46, 52, 210127), None, 'quay', 'quay', datetime.datetime(2023, 8, 8, 5, 46, 52, 210286), 'quay', None, datetime.datetime(2023, 8, 7, 5, 46, 52, 210312), datetime.datetime(2023, 8, 1, 5, 46, 52, 210326), None]) gunicorn-web stdout | 2023-08-08 05:46:52,215 [199] [DEBUG] [app] Ending request: urn:request:341eafdf-a26b-4b45-bf55-498728fd5847 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:341eafdf-a26b-4b45-bf55-498728fd5847', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:46:52,215 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:46:52,215 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:46:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:46:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.038 47 0.039) gunicorn-web stdout | 2023-08-08 05:46:52,220 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:46:52,231 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:46:52,233 [200] [DEBUG] [app] Ending request: urn:request:7bc1949a-ff85-439d-ae6e-80d46b5f3991 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7bc1949a-ff85-439d-ae6e-80d46b5f3991', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:46:52,233 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:46:52,233 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:46:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:46:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.056 47 0.056) chunkcleanupworker stdout | 2023-08-08 05:46:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:46:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:02.293757+00:00 (in 9.999261 seconds) chunkcleanupworker stdout | 2023-08-08 05:46:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:02 UTC)" (scheduled at 2023-08-08 05:46:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:46:52,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:46:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 52, 295119), True, datetime.datetime(2023, 8, 8, 5, 46, 52, 295119), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:46:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:46:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:46:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:46:52,335 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:46:54,114 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:46:54,236 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} securityscanningnotificationworker stdout | 2023-08-08 05:46:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:46:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:13.097375+00:00 (in 18.812731 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:46:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:54 UTC)" (scheduled at 2023-08-08 05:46:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:46:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:46:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:46:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:46:54,395 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:36.579841+00:00 (in 42.184787 seconds) namespacegcworker stdout | 2023-08-08 05:46:54,395 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:54 UTC)" (scheduled at 2023-08-08 05:46:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:46:54,395 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:46:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 54, 395596), True, datetime.datetime(2023, 8, 8, 5, 46, 54, 395596), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:46:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:46:54,404 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:46:54,404 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:47:54 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:46:54,508 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:46:54,612 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:46:54,615 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:46:54,618 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:46:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:46:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:02.178530+00:00 (in 6.806334 seconds) notificationworker stdout | 2023-08-08 05:46:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:05 UTC)" (scheduled at 2023-08-08 05:46:55.371682+00:00) notificationworker stdout | 2023-08-08 05:46:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:46:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 46, 55, 372732), True, datetime.datetime(2023, 8, 8, 5, 46, 55, 372732), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:46:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:46:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:46:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:46:55,550 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:46:56,112 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:46:56,327 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:46:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:46:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:19.687363+00:00 (in 22.796105 seconds) securityworker stdout | 2023-08-08 05:46:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:26 UTC)" (scheduled at 2023-08-08 05:46:56.890741+00:00) securityworker stdout | 2023-08-08 05:46:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:46:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:46:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:46:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:46:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:46:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:46:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:46:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:46:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:46:56,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:46:56,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:46:56,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:46:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:46:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:46:56,904 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:46:56,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:46:56,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:46:56,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:46:56,988 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:46:56,992 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:46:57,003 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:46:57,005 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:46:57,008 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:46:57,549 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:46:57,637 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:46:58,747 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} builder stdout | 2023-08-08 05:46:59,417 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue gcworker stdout | 2023-08-08 05:46:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:46:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:09.417660+00:00 (in 9.999484 seconds) gcworker stdout | 2023-08-08 05:46:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:09 UTC)" (scheduled at 2023-08-08 05:46:59.417660+00:00) gcworker stdout | 2023-08-08 05:46:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264019418, 1209600, True, 3, 500, 1, 0]) builder stdout | 2023-08-08 05:46:59,427 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:46:59,427 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics gcworker stdout | 2023-08-08 05:46:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:46:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:46:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:09 UTC)" executed successfully builder stdout | 2023-08-08 05:46:59,431 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:46:59,540 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:46:59,623 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:47:00,148 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:47:00,738 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:47:01,010 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:47:01,241 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:47:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:47:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:05.371682+00:00 (in 3.192593 seconds) notificationworker stdout | 2023-08-08 05:47:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:02 UTC)" (scheduled at 2023-08-08 05:47:02.178530+00:00) notificationworker stdout | 2023-08-08 05:47:02,180 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:47:02,180 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:47:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:47:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:12.293757+00:00 (in 9.999433 seconds) chunkcleanupworker stdout | 2023-08-08 05:47:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:12 UTC)" (scheduled at 2023-08-08 05:47:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:47:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:47:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 2, 294942), True, datetime.datetime(2023, 8, 8, 5, 47, 2, 294942), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:47:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:47:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:47:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:47:02,792 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:47:02,795 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:47:02,798 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:47:02,800 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:47:02,803 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:47:02,805 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:47:02,808 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:47:02,811 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 05:47:02,813 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 05:47:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:47:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:09.793658+00:00 (in 5.605707 seconds) repositorygcworker stdout | 2023-08-08 05:47:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:04 UTC)" (scheduled at 2023-08-08 05:47:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:47:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:47:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:47:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:47:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:15.371682+00:00 (in 9.999482 seconds) notificationworker stdout | 2023-08-08 05:47:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:15 UTC)" (scheduled at 2023-08-08 05:47:05.371682+00:00) notificationworker stdout | 2023-08-08 05:47:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:47:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 5, 373086), True, datetime.datetime(2023, 8, 8, 5, 47, 5, 373086), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:47:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:47:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:47:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:47:07,178 [197] [DEBUG] [app] Starting request: urn:request:a586597e-4175-4783-9ddc-030c4ce2019d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:47:07,179 [200] [DEBUG] [app] Starting request: urn:request:686ca399-7f8d-4e3c-821f-6bd084323de3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:47:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:47:07,184 [204] [DEBUG] [app] Starting request: urn:request:58f9e662-9cc9-41e9-8e2f-7b8ffbebfe08 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:47:07,184 [206] [DEBUG] [app] Starting request: urn:request:eb891577-973b-4330-8b91-df3a541fcda3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:47:07,185 [204] [DEBUG] [app] Ending request: urn:request:58f9e662-9cc9-41e9-8e2f-7b8ffbebfe08 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:58f9e662-9cc9-41e9-8e2f-7b8ffbebfe08', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:47:07,185 [206] [DEBUG] [app] Ending request: urn:request:eb891577-973b-4330-8b91-df3a541fcda3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:eb891577-973b-4330-8b91-df3a541fcda3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:47:07,185 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:47:07,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:47:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:07,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:07,191 [200] [DEBUG] [app] Starting request: urn:request:0403f87a-1654-4ab4-a230-59746f822244 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:07,191 [200] [DEBUG] [app] Ending request: urn:request:0403f87a-1654-4ab4-a230-59746f822244 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0403f87a-1654-4ab4-a230-59746f822244', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:47:07,192 [199] [DEBUG] [app] Starting request: urn:request:0dbf5e5a-7889-496b-9dd1-d9ceef37a8f9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:47:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:07,192 [199] [DEBUG] [app] Ending request: urn:request:0dbf5e5a-7889-496b-9dd1-d9ceef37a8f9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0dbf5e5a-7889-496b-9dd1-d9ceef37a8f9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:47:07,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:47:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:47:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:47:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:47:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:37.197412+00:00 (in 29.999524 seconds) buildlogsarchiver stdout | 2023-08-08 05:47:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:37 UTC)" (scheduled at 2023-08-08 05:47:07.197412+00:00) gunicorn-web stdout | 2023-08-08 05:47:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:47:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 47, 7, 198365), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:47:07,203 [197] [DEBUG] [app] Ending request: urn:request:a586597e-4175-4783-9ddc-030c4ce2019d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a586597e-4175-4783-9ddc-030c4ce2019d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:47:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:47:07,204 [200] [DEBUG] [app] Ending request: urn:request:686ca399-7f8d-4e3c-821f-6bd084323de3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:686ca399-7f8d-4e3c-821f-6bd084323de3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:47:07,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:47:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:47:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:47:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:47:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:47:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:47:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 05:47:07,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:47:07,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:47:07,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:37 UTC)" executed successfully gcworker stdout | 2023-08-08 05:47:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:47:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:19.417660+00:00 (in 9.999437 seconds) gcworker stdout | 2023-08-08 05:47:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:19 UTC)" (scheduled at 2023-08-08 05:47:09.417660+00:00) gcworker stdout | 2023-08-08 05:47:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264029418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:47:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:47:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:47:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:19 UTC)" executed successfully builder stdout | 2023-08-08 05:47:09,432 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:47:09,441 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:47:09,441 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:47:09,446 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:47:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:47:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:04.187429+00:00 (in 54.393223 seconds) repositorygcworker stdout | 2023-08-08 05:47:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:09 UTC)" (scheduled at 2023-08-08 05:47:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:47:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:47:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 9, 794667), True, datetime.datetime(2023, 8, 8, 5, 47, 9, 794667), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:47:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:47:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:47:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:47:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:47:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:22.293757+00:00 (in 9.999446 seconds) chunkcleanupworker stdout | 2023-08-08 05:47:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:22 UTC)" (scheduled at 2023-08-08 05:47:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:47:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:47:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 12, 294973), True, datetime.datetime(2023, 8, 8, 5, 47, 12, 294973), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:47:12,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:47:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:47:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:47:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:47:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:54.284093+00:00 (in 41.186182 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:47:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:13 UTC)" (scheduled at 2023-08-08 05:47:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:47:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:47:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 13, 98502), True, datetime.datetime(2023, 8, 8, 5, 47, 13, 98502), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:47:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:47:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:47:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:47:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:47:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:25.371682+00:00 (in 9.999501 seconds) notificationworker stdout | 2023-08-08 05:47:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:25 UTC)" (scheduled at 2023-08-08 05:47:15.371682+00:00) notificationworker stdout | 2023-08-08 05:47:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:47:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 15, 372754), True, datetime.datetime(2023, 8, 8, 5, 47, 15, 372754), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:47:15,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:47:15,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:47:15,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:47:16,739 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:47:17,950 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:47:19,121 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:47:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:47:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:29.417660+00:00 (in 9.999385 seconds) gcworker stdout | 2023-08-08 05:47:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:29 UTC)" (scheduled at 2023-08-08 05:47:19.417660+00:00) gcworker stdout | 2023-08-08 05:47:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264039418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:47:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:47:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:47:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:29 UTC)" executed successfully builder stdout | 2023-08-08 05:47:19,452 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:47:19,460 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:47:19,460 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:47:19,465 [55] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2023-08-08 05:47:19,579 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityworker stdout | 2023-08-08 05:47:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:47:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:26.890741+00:00 (in 7.202867 seconds) securityworker stdout | 2023-08-08 05:47:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:49 UTC)" (scheduled at 2023-08-08 05:47:19.687363+00:00) securityworker stdout | 2023-08-08 05:47:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:47:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:47:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:47:19,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:47:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:47:19,700 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:47:19,700 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:49 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:47:22,178 [197] [DEBUG] [app] Starting request: urn:request:0e906099-0c4b-486c-9c42-508294042240 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:47:22,179 [199] [DEBUG] [app] Starting request: urn:request:0f350d3e-cb4c-4bd1-a2d9-efb2337536bd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:47:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:22,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:47:22,185 [208] [DEBUG] [app] Starting request: urn:request:31971159-856b-495e-99f9-8246524373ac (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:47:22,185 [208] [DEBUG] [app] Ending request: urn:request:31971159-856b-495e-99f9-8246524373ac (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:31971159-856b-495e-99f9-8246524373ac', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:47:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:47:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:47:22,186 [203] [DEBUG] [app] Starting request: urn:request:870fa466-8cb9-4632-9630-6ccf1af043b9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:47:22,188 [203] [DEBUG] [app] Ending request: urn:request:870fa466-8cb9-4632-9630-6ccf1af043b9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:870fa466-8cb9-4632-9630-6ccf1af043b9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.005 162 0.005) gunicorn-registry stdout | 2023-08-08 05:47:22,188 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:47:22,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:22,191 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:22,191 [200] [DEBUG] [app] Starting request: urn:request:af72d6bd-dfd4-435a-ba78-98f1687edc7b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:22,192 [200] [DEBUG] [app] Ending request: urn:request:af72d6bd-dfd4-435a-ba78-98f1687edc7b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:af72d6bd-dfd4-435a-ba78-98f1687edc7b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:47:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:22,193 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:22,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:47:22,194 [198] [DEBUG] [app] Starting request: urn:request:7bbd90bb-634e-44b4-aa93-67d31f3c267d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:22,194 [198] [DEBUG] [app] Ending request: urn:request:7bbd90bb-634e-44b4-aa93-67d31f3c267d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7bbd90bb-634e-44b4-aa93-67d31f3c267d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:47:22,195 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:47:22,195 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:22,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:47:22,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:47:22,200 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:47:22,205 [199] [DEBUG] [app] Ending request: urn:request:0f350d3e-cb4c-4bd1-a2d9-efb2337536bd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0f350d3e-cb4c-4bd1-a2d9-efb2337536bd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:47:22,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:47:22,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:47:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:47:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) gunicorn-web stdout | 2023-08-08 05:47:22,206 [197] [DEBUG] [app] Ending request: urn:request:0e906099-0c4b-486c-9c42-508294042240 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0e906099-0c4b-486c-9c42-508294042240', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:47:22,206 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:47:22,207 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:47:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:47:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.030 47 0.029) chunkcleanupworker stdout | 2023-08-08 05:47:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:47:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:28.874906+00:00 (in 6.580579 seconds) chunkcleanupworker stdout | 2023-08-08 05:47:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:32 UTC)" (scheduled at 2023-08-08 05:47:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:47:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:47:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 22, 294957), True, datetime.datetime(2023, 8, 8, 5, 47, 22, 294957), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:47:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:47:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:47:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:47:22,354 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:47:24,132 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:47:24,272 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:47:24,515 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:47:24,621 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:47:24,624 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:47:24,628 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:47:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:47:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:35.371682+00:00 (in 9.999440 seconds) notificationworker stdout | 2023-08-08 05:47:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:35 UTC)" (scheduled at 2023-08-08 05:47:25.371682+00:00) notificationworker stdout | 2023-08-08 05:47:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:47:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 25, 372773), True, datetime.datetime(2023, 8, 8, 5, 47, 25, 372773), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:47:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:47:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:47:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:47:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:47:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:47.398551+00:00 (in 21.883490 seconds) exportactionlogsworker stdout | 2023-08-08 05:47:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:25 UTC)" (scheduled at 2023-08-08 05:47:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:47:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:47:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:47:25,586 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:47:26,132 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:47:26,364 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:47:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:47:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:49.687363+00:00 (in 22.796105 seconds) securityworker stdout | 2023-08-08 05:47:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:56 UTC)" (scheduled at 2023-08-08 05:47:26.890741+00:00) securityworker stdout | 2023-08-08 05:47:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:47:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:47:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:47:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:47:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:47:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:47:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:47:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:47:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:47:26,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:47:26,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:47:26,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:47:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:47:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:47:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:47:26,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:47:26,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:47:26,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:47:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:47:26,997 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:47:27,001 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:47:27,009 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:47:27,012 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:47:27,014 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:47:27,571 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:47:27,674 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:47:28,785 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 05:47:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:47:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:32.293757+00:00 (in 3.418287 seconds) chunkcleanupworker stdout | 2023-08-08 05:47:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:28 UTC)" (scheduled at 2023-08-08 05:47:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:47:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:47:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:28 UTC)" executed successfully gcworker stdout | 2023-08-08 05:47:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:47:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:39.417660+00:00 (in 9.999376 seconds) gcworker stdout | 2023-08-08 05:47:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:39 UTC)" (scheduled at 2023-08-08 05:47:29.417660+00:00) gcworker stdout | 2023-08-08 05:47:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264049418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:47:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:47:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:47:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:39 UTC)" executed successfully builder stdout | 2023-08-08 05:47:29,475 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:47:29,483 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:47:29,484 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:47:29,488 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:47:29,576 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:47:29,630 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:47:30,171 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:47:30,754 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:47:31,019 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:47:31,277 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:47:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:47:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:42.293757+00:00 (in 9.999443 seconds) chunkcleanupworker stdout | 2023-08-08 05:47:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:42 UTC)" (scheduled at 2023-08-08 05:47:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:47:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:47:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 32, 294804), True, datetime.datetime(2023, 8, 8, 5, 47, 32, 294804), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:47:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:47:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:47:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:47:32,801 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:47:32,807 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:47:32,809 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:47:32,812 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:47:32,815 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:47:32,818 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:47:32,820 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:47:32,822 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:47:32,825 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 05:47:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:47:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:45.371682+00:00 (in 9.999480 seconds) notificationworker stdout | 2023-08-08 05:47:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:45 UTC)" (scheduled at 2023-08-08 05:47:35.371682+00:00) notificationworker stdout | 2023-08-08 05:47:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:47:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 35, 372730), True, datetime.datetime(2023, 8, 8, 5, 47, 35, 372730), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:47:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:47:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:47:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:47:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:47:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:54.393627+00:00 (in 17.813263 seconds) namespacegcworker stdout | 2023-08-08 05:47:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:36 UTC)" (scheduled at 2023-08-08 05:47:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:47:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:47:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:47:37,179 [200] [DEBUG] [app] Starting request: urn:request:9a59c767-81c1-4575-9474-3c182a808d49 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:47:37,179 [197] [DEBUG] [app] Starting request: urn:request:3c8563da-ec0c-4912-bbae-2178910ccd1f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:47:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:47:37,185 [206] [DEBUG] [app] Starting request: urn:request:1f3d6459-ae7d-436b-afb5-03ad39b09fc0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:47:37,186 [206] [DEBUG] [app] Ending request: urn:request:1f3d6459-ae7d-436b-afb5-03ad39b09fc0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1f3d6459-ae7d-436b-afb5-03ad39b09fc0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:47:37,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:47:37,186 [210] [DEBUG] [app] Starting request: urn:request:db67c5bf-d0b0-429b-97a5-3780b241c666 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:47:37,188 [210] [DEBUG] [app] Ending request: urn:request:db67c5bf-d0b0-429b-97a5-3780b241c666 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:db67c5bf-d0b0-429b-97a5-3780b241c666', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.005 162 0.005) gunicorn-registry stdout | 2023-08-08 05:47:37,188 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:47:37,188 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:37,190 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:37,191 [198] [DEBUG] [app] Starting request: urn:request:45a74b0e-a39b-4388-b991-3fcf151a5dd7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:37,192 [198] [DEBUG] [app] Ending request: urn:request:45a74b0e-a39b-4388-b991-3fcf151a5dd7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:45a74b0e-a39b-4388-b991-3fcf151a5dd7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:37,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:47:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:47:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:47:37,193 [199] [DEBUG] [app] Starting request: urn:request:b6d139e5-6769-43ae-ba73-0641669db364 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:37,194 [199] [DEBUG] [app] Ending request: urn:request:b6d139e5-6769-43ae-ba73-0641669db364 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b6d139e5-6769-43ae-ba73-0641669db364', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:37,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:47:37,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:37,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:47:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:47:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:07.197412+00:00 (in 29.999538 seconds) buildlogsarchiver stdout | 2023-08-08 05:47:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:07 UTC)" (scheduled at 2023-08-08 05:47:37.197412+00:00) gunicorn-web stdout | 2023-08-08 05:47:37,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:47:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 47, 37, 198306), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:47:37,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:47:37,204 [197] [DEBUG] [app] Ending request: urn:request:3c8563da-ec0c-4912-bbae-2178910ccd1f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3c8563da-ec0c-4912-bbae-2178910ccd1f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:47:37,205 [197] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:47:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:47:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:47:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:47:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:47:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:47:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:47:37,205 [200] [DEBUG] [app] Ending request: urn:request:9a59c767-81c1-4575-9474-3c182a808d49 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9a59c767-81c1-4575-9474-3c182a808d49', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:47:37,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:47:37,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:47:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:47:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) gcworker stdout | 2023-08-08 05:47:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:47:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:49.417660+00:00 (in 9.999440 seconds) gcworker stdout | 2023-08-08 05:47:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:49 UTC)" (scheduled at 2023-08-08 05:47:39.417660+00:00) gcworker stdout | 2023-08-08 05:47:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264059418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:47:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:47:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:47:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:49 UTC)" executed successfully builder stdout | 2023-08-08 05:47:39,498 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:47:39,506 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:47:39,506 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:47:39,511 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:47:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:47:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:52.293757+00:00 (in 9.999502 seconds) chunkcleanupworker stdout | 2023-08-08 05:47:42,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:52 UTC)" (scheduled at 2023-08-08 05:47:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:47:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:47:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 42, 295351), True, datetime.datetime(2023, 8, 8, 5, 47, 42, 295351), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:47:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:47:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:47:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:47:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:47:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:55.371682+00:00 (in 9.999427 seconds) notificationworker stdout | 2023-08-08 05:47:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:55 UTC)" (scheduled at 2023-08-08 05:47:45.371682+00:00) notificationworker stdout | 2023-08-08 05:47:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:47:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 45, 372816), True, datetime.datetime(2023, 8, 8, 5, 47, 45, 372816), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:47:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:47:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:47:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:47:46,775 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:47:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:47:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:25.514341+00:00 (in 38.115246 seconds) exportactionlogsworker stdout | 2023-08-08 05:47:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:47 UTC)" (scheduled at 2023-08-08 05:47:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:47:47,400 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:47:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 47, 400083), True, datetime.datetime(2023, 8, 8, 5, 47, 47, 400083), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:47:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:47:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:47:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:47:47,960 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:47:49,158 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:47:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:47:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:59.417660+00:00 (in 9.999474 seconds) gcworker stdout | 2023-08-08 05:47:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:59 UTC)" (scheduled at 2023-08-08 05:47:49.417660+00:00) gcworker stdout | 2023-08-08 05:47:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264069418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:47:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:47:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:47:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:47:59 UTC)" executed successfully builder stdout | 2023-08-08 05:47:49,520 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:47:49,529 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:47:49,529 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:47:49,533 [55] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2023-08-08 05:47:49,608 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityworker stdout | 2023-08-08 05:47:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:47:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:47:56.890741+00:00 (in 7.202842 seconds) securityworker stdout | 2023-08-08 05:47:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:19 UTC)" (scheduled at 2023-08-08 05:47:49.687363+00:00) securityworker stdout | 2023-08-08 05:47:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:47:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:47:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:47:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:47:49,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:47:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:47:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:19 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:47:52,179 [199] [DEBUG] [app] Starting request: urn:request:bdda7430-5586-422f-b519-48b8dfa4f621 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:47:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:52,180 [197] [DEBUG] [app] Starting request: urn:request:7c5fa034-13c7-47e5-a1ae-d7d85126480b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:47:52,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:47:52,185 [209] [DEBUG] [app] Starting request: urn:request:be7ba30d-de07-446b-9bcc-d26243e85dad (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:47:52,185 [203] [DEBUG] [app] Starting request: urn:request:c00d41c3-405a-4960-af09-1f84d8e10981 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:47:52,185 [209] [DEBUG] [app] Ending request: urn:request:be7ba30d-de07-446b-9bcc-d26243e85dad (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:be7ba30d-de07-446b-9bcc-d26243e85dad', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:47:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:47:52,186 [203] [DEBUG] [app] Ending request: urn:request:c00d41c3-405a-4960-af09-1f84d8e10981 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c00d41c3-405a-4960-af09-1f84d8e10981', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:47:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 05:47:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:52,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:47:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:47:52,191 [197] [DEBUG] [app] Starting request: urn:request:e181e7a8-3ba3-4c52-a015-ab2ff9400c8b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:52,191 [200] [DEBUG] [app] Starting request: urn:request:3827f697-af97-4d94-8121-6c9813e4de53 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:47:52,192 [197] [DEBUG] [app] Ending request: urn:request:e181e7a8-3ba3-4c52-a015-ab2ff9400c8b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e181e7a8-3ba3-4c52-a015-ab2ff9400c8b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:52,192 [200] [DEBUG] [app] Ending request: urn:request:3827f697-af97-4d94-8121-6c9813e4de53 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3827f697-af97-4d94-8121-6c9813e4de53', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:47:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 05:47:52,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:47:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:47:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:47:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:47:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:47:52,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:47:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:47:52,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:47:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:47:52,203 [197] [DEBUG] [app] Ending request: urn:request:7c5fa034-13c7-47e5-a1ae-d7d85126480b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7c5fa034-13c7-47e5-a1ae-d7d85126480b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:47:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:47:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:47:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:47:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 05:47:52,204 [199] [DEBUG] [app] Ending request: urn:request:bdda7430-5586-422f-b519-48b8dfa4f621 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bdda7430-5586-422f-b519-48b8dfa4f621', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:47:52,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:47:52,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:47:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:47:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) chunkcleanupworker stdout | 2023-08-08 05:47:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:47:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:02.293757+00:00 (in 9.999399 seconds) chunkcleanupworker stdout | 2023-08-08 05:47:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:02 UTC)" (scheduled at 2023-08-08 05:47:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:47:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:47:52,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 52, 294978), True, datetime.datetime(2023, 8, 8, 5, 47, 52, 294978), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:47:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:47:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:47:52,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:47:52,388 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:47:54,169 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityscanningnotificationworker stdout | 2023-08-08 05:47:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:47:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:13.097375+00:00 (in 18.812699 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:47:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:54 UTC)" (scheduled at 2023-08-08 05:47:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:47:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:47:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:54 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:47:54,303 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} namespacegcworker stdout | 2023-08-08 05:47:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:47:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:36.579841+00:00 (in 42.185670 seconds) namespacegcworker stdout | 2023-08-08 05:47:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:54 UTC)" (scheduled at 2023-08-08 05:47:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:47:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:47:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 54, 394724), True, datetime.datetime(2023, 8, 8, 5, 47, 54, 394724), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:47:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:47:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:47:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:48:54 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:47:54,528 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:47:54,630 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:47:54,633 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:47:54,635 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:47:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:47:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:02.178530+00:00 (in 6.806308 seconds) notificationworker stdout | 2023-08-08 05:47:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:05 UTC)" (scheduled at 2023-08-08 05:47:55.371682+00:00) notificationworker stdout | 2023-08-08 05:47:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:47:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 47, 55, 372710), True, datetime.datetime(2023, 8, 8, 5, 47, 55, 372710), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:47:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:47:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:47:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:47:55,616 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:47:56,164 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:47:56,398 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:47:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:47:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:19.687363+00:00 (in 22.796045 seconds) securityworker stdout | 2023-08-08 05:47:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:26 UTC)" (scheduled at 2023-08-08 05:47:56.890741+00:00) securityworker stdout | 2023-08-08 05:47:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:47:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:47:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:47:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:47:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:47:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:47:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:47:56,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:47:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:47:56,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:47:56,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:47:56,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:47:56,902 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:47:56,902 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:47:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:47:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:47:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:47:56,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:47:57,010 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:47:57,012 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:47:57,015 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:47:57,021 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:47:57,024 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:47:57,608 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:47:57,711 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:47:58,821 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:47:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:47:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:09.417660+00:00 (in 9.999443 seconds) gcworker stdout | 2023-08-08 05:47:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:09 UTC)" (scheduled at 2023-08-08 05:47:59.417660+00:00) gcworker stdout | 2023-08-08 05:47:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264079418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:47:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:47:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:47:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:09 UTC)" executed successfully builder stdout | 2023-08-08 05:47:59,544 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:47:59,552 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:47:59,553 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:47:59,557 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:47:59,613 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:47:59,656 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:48:00,194 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:48:00,766 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:48:01,037 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:48:01,313 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:48:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:48:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:05.371682+00:00 (in 3.192625 seconds) notificationworker stdout | 2023-08-08 05:48:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:02 UTC)" (scheduled at 2023-08-08 05:48:02.178530+00:00) notificationworker stdout | 2023-08-08 05:48:02,180 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:48:02,180 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:48:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:48:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:12.293757+00:00 (in 9.999364 seconds) chunkcleanupworker stdout | 2023-08-08 05:48:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:12 UTC)" (scheduled at 2023-08-08 05:48:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:48:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:48:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 2, 294952), True, datetime.datetime(2023, 8, 8, 5, 48, 2, 294952), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:48:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:48:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:48:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:48:02,814 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:48:02,817 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:48:02,819 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:48:02,824 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:48:02,827 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:48:02,829 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:48:02,832 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:48:02,835 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:48:02,839 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:48:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:48:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:09.793658+00:00 (in 5.605712 seconds) repositorygcworker stdout | 2023-08-08 05:48:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:04 UTC)" (scheduled at 2023-08-08 05:48:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:48:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:48:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:48:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:48:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:15.371682+00:00 (in 9.999427 seconds) notificationworker stdout | 2023-08-08 05:48:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:15 UTC)" (scheduled at 2023-08-08 05:48:05.371682+00:00) notificationworker stdout | 2023-08-08 05:48:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:48:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 5, 372922), True, datetime.datetime(2023, 8, 8, 5, 48, 5, 372922), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:48:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:48:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:48:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:07,180 [197] [DEBUG] [app] Starting request: urn:request:6d2df632-d0e8-4d61-86aa-f11beb8836f2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:48:07,180 [200] [DEBUG] [app] Starting request: urn:request:0496bf22-e057-4e09-85d3-3b9325af0fdc (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:48:07,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:07,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:07,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:07,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:48:07,186 [203] [DEBUG] [app] Starting request: urn:request:43bb9ab6-fac2-4522-8365-4278ff653f50 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:48:07,186 [206] [DEBUG] [app] Starting request: urn:request:8b374d17-b18a-44db-8641-ac10ec61f8c2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:48:07,186 [203] [DEBUG] [app] Ending request: urn:request:43bb9ab6-fac2-4522-8365-4278ff653f50 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:43bb9ab6-fac2-4522-8365-4278ff653f50', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:48:07,187 [206] [DEBUG] [app] Ending request: urn:request:8b374d17-b18a-44db-8641-ac10ec61f8c2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8b374d17-b18a-44db-8641-ac10ec61f8c2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:48:07,187 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:48:07,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:07,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:07,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:07,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:07,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:07,192 [200] [DEBUG] [app] Starting request: urn:request:2994975f-2b32-4a03-8870-c3b3776949e9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:48:07,193 [200] [DEBUG] [app] Ending request: urn:request:2994975f-2b32-4a03-8870-c3b3776949e9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2994975f-2b32-4a03-8870-c3b3776949e9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:48:07,193 [198] [DEBUG] [app] Starting request: urn:request:f33421de-cc9e-483e-bbd2-25bcd0cbdbf5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:48:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:07,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:07,194 [198] [DEBUG] [app] Ending request: urn:request:f33421de-cc9e-483e-bbd2-25bcd0cbdbf5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f33421de-cc9e-483e-bbd2-25bcd0cbdbf5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:48:07,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:48:07,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:07,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:48:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:48:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:37.197412+00:00 (in 29.999493 seconds) buildlogsarchiver stdout | 2023-08-08 05:48:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:37 UTC)" (scheduled at 2023-08-08 05:48:07.197412+00:00) gunicorn-web stdout | 2023-08-08 05:48:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:48:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 48, 7, 198854), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:07,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:48:07,204 [200] [DEBUG] [app] Ending request: urn:request:0496bf22-e057-4e09-85d3-3b9325af0fdc (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0496bf22-e057-4e09-85d3-3b9325af0fdc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:48:07,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:48:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 05:48:07,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:48:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:48:07,205 [197] [DEBUG] [app] Ending request: urn:request:6d2df632-d0e8-4d61-86aa-f11beb8836f2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6d2df632-d0e8-4d61-86aa-f11beb8836f2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:48:07,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:07,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:48:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:48:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) buildlogsarchiver stdout | 2023-08-08 05:48:07,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:48:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:48:07,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:37 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:09,072 [197] [DEBUG] [app] Starting request: urn:request:26f743b7-aab0-46ea-981d-4a9043b960bb (/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:09,081 [197] [DEBUG] [endpoints.api.discovery] Unable to find method for POST in class gunicorn-web stdout | 2023-08-08 05:48:09,112 [197] [DEBUG] [app] Ending request: urn:request:26f743b7-aab0-46ea-981d-4a9043b960bb (/) {'endpoint': 'web.index', 'request_id': 'urn:request:26f743b7-aab0-46ea-981d-4a9043b960bb', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/', 'path': '/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:09,113 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:09 +0000] "GET / HTTP/1.0" 200 85192 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET / HTTP/2.0" 200 13318 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.044 724 0.044) gcworker stdout | 2023-08-08 05:48:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:48:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:19.417660+00:00 (in 9.999430 seconds) gcworker stdout | 2023-08-08 05:48:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:19 UTC)" (scheduled at 2023-08-08 05:48:09.417660+00:00) gcworker stdout | 2023-08-08 05:48:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264089418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:48:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:48:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:48:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:19 UTC)" executed successfully builder stdout | 2023-08-08 05:48:09,567 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:48:09,576 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:48:09,576 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:48:09,581 [55] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/font-awesome.css-a166e3c966c6.css HTTP/2.0" 200 8404 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 298 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/bootstrap.min.css-13f668494557.css HTTP/2.0" 200 24427 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 66 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/cssfamilySourceSansPro300400700-30cd793a4b9b.css HTTP/2.0" 200 224 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 76 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/bootstrap-datetimepicker.min.css-4b15a4ede750.css HTTP/2.0" 200 1470 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 76 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/cal-heatmap.css-904c3b9e8a60.css HTTP/2.0" 200 787 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 65 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/ng-tags-input.min.css-7782190f81cd.css HTTP/2.0" 200 1067 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 69 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/build/main-quay-frontend-ab93c42f539d59de0115.bundle.js HTTP/2.0" 200 719930 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.059 90 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/img/RedHat.svg HTTP/2.0" 200 1233 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.022 77 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/jquery.js-3c5c5966471e.js HTTP/2.0" 200 102774 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.022 102 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/bootstrap.min.js-0e773844b374.js HTTP/2.0" 200 11290 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.022 83 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/angular.min.js-f0f4228b82f7.js HTTP/2.0" 200 63479 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 82 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/angular-route.min.js-590b72b0eb88.js HTTP/2.0" 200 2236 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 86 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/angular-sanitize.min.js-ff589de0ec1b.js HTTP/2.0" 200 3116 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 88 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/angular-animate.min.js-636f7a51b2e1.js HTTP/2.0" 200 10296 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 88 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/angular-cookies.min.js-2455ea25d522.js HTTP/2.0" 200 814 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 86 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/momentjs-66a52705d33f.js HTTP/2.0" 200 19464 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 78 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/bootstrap-datepicker.min.js-3e3d8e141d5e.js HTTP/2.0" 200 9310 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 91 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/bootstrap-datetimepicker.min.js-6612bb423b71.js HTTP/2.0" 200 10819 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 94 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/bootbox4.1.0underscorejs1.5.2restangular1.2.0d3js3.3.3-6d47ddfef9c5.js HTTP/2.0" 200 71170 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 110 -) repositorygcworker stdout | 2023-08-08 05:48:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:48:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:04.187429+00:00 (in 54.393271 seconds) repositorygcworker stdout | 2023-08-08 05:48:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:09 UTC)" (scheduled at 2023-08-08 05:48:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:48:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:48:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 9, 795117), True, datetime.datetime(2023, 8, 8, 5, 48, 9, 795117), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:48:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:48:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:48:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:09 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/raven.min.js-ecd480647a39.js HTTP/2.0" 200 9492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 81 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/cal-heatmap.min.js-8ffad728269a.js HTTP/2.0" 200 12991 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 85 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/angular-recaptcha.min.js-23859fca3602.js HTTP/2.0" 200 1781 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 88 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/ng-tags-input.min.js-fa3026ba8409.js HTTP/2.0" 200 6089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 86 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/ldn/typeahead.bundle.min.js-716151ca4e5d.js HTTP/2.0" 200 16660 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 88 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:09 +0000] "GET /static/standalonelib/jquery.overscroll.min.js HTTP/2.0" 200 3479 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 85 -) chunkcleanupworker stdout | 2023-08-08 05:48:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:48:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:22.293757+00:00 (in 9.999424 seconds) chunkcleanupworker stdout | 2023-08-08 05:48:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:22 UTC)" (scheduled at 2023-08-08 05:48:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:48:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:48:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 12, 294953), True, datetime.datetime(2023, 8, 8, 5, 48, 12, 294953), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:48:12,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:48:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:48:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:22 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/build/83-quay-frontend.chunk.js HTTP/2.0" 200 1705 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 63 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/build/135-quay-frontend.chunk.js HTTP/2.0" 200 963 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 63 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/build/18-quay-frontend.chunk.js HTTP/2.0" 200 985 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 62 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/build/112-quay-frontend.chunk.js HTTP/2.0" 200 755 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 63 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/build/181-quay-frontend.chunk.js HTTP/2.0" 200 940 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 104 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/build/152-quay-frontend.chunk.js HTTP/2.0" 200 313 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 78 -) securityscanningnotificationworker stdout | 2023-08-08 05:48:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:48:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:54.284093+00:00 (in 41.186194 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:48:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:13 UTC)" (scheduled at 2023-08-08 05:48:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:48:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:48:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 13, 98380), True, datetime.datetime(2023, 8, 8, 5, 48, 13, 98380), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:48:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:48:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:48:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:13 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/img/network-tile.png HTTP/2.0" 200 6289 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 54 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/build/489e5e00e1a9a80f88d55281894a87ff.woff HTTP/2.0" 200 7280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 139 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/fonts/fontawesome-webfont.woff2?v=4.7.0 HTTP/2.0" 200 77160 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/static/ldn/font-awesome.css-a166e3c966c6.css" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 163 -) gunicorn-web stdout | 2023-08-08 05:48:13,212 [199] [DEBUG] [app] Starting request: urn:request:30945880-d873-43ee-815e-c716a0651032 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:13,213 [199] [DEBUG] [app] Ending request: urn:request:30945880-d873-43ee-815e-c716a0651032 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:30945880-d873-43ee-815e-c716a0651032', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /api/v1/user/ HTTP/2.0" 401 306 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.004 167 0.004) gunicorn-web stdout | 2023-08-08 05:48:13,214 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:13 +0000] "GET /api/v1/user/ HTTP/1.0" 401 306 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:13,308 [199] [DEBUG] [app] Starting request: urn:request:0600c10e-49e9-4f82-8f9b-20e255623367 (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:13,309 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /static/img/RH_Logo_Quay_Black_UX-horizontal.svg HTTP/2.0" 200 3210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 70 -) gunicorn-web stdout | 2023-08-08 05:48:13,316 [199] [DEBUG] [app] Ending request: urn:request:0600c10e-49e9-4f82-8f9b-20e255623367 (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:0600c10e-49e9-4f82-8f9b-20e255623367', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:13,317 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:13,317 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:13 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:13 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 157 0.011) notificationworker stdout | 2023-08-08 05:48:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:48:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:25.371682+00:00 (in 9.999156 seconds) notificationworker stdout | 2023-08-08 05:48:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:25 UTC)" (scheduled at 2023-08-08 05:48:15.371682+00:00) notificationworker stdout | 2023-08-08 05:48:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:48:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 15, 373028), True, datetime.datetime(2023, 8, 8, 5, 48, 15, 373028), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:48:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:48:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:48:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:16,207 [199] [DEBUG] [app] Starting request: urn:request:dd262ac1-5596-4692-8f18-c4fb85202cb9 (/api/v1/signin) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:16,210 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['whuquay', 'whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:16,489 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:16,492 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:16,493 [199] [DEBUG] [endpoints.common] Successfully signed in as user whuquay with uuid fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,493 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:16,494 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,494 [199] [DEBUG] [app] Ending request: urn:request:dd262ac1-5596-4692-8f18-c4fb85202cb9 (/api/v1/signin) {'endpoint': 'api.signin', 'request_id': 'urn:request:dd262ac1-5596-4692-8f18-c4fb85202cb9', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/signin', 'path': '/api/v1/signin', 'parameters': {}, 'json_body': {'username': 'whuquay', 'password': '[FILTERED]'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:16,495 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:16,495 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:16 +0000] "POST /api/v1/signin HTTP/1.0" 200 18 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:16 +0000] "POST /api/v1/signin HTTP/2.0" 200 18 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.289 338 0.290) notificationworker stdout | 2023-08-08 05:48:16,783 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gunicorn-web stdout | 2023-08-08 05:48:16,847 [200] [DEBUG] [app] Starting request: urn:request:e4980369-8fb2-4bce-bf6d-255ae2a31418 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:16,847 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,847 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,848 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:16,855 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:16,855 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:16,856 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,856 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,856 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:16,856 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:16,856 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:16,858 [200] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 05:48:16,860 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,861 [200] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 05:48:16,863 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:48:16,864 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:16,866 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:48:16,867 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:16,869 [200] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691473696868, 0, 1, None]) gunicorn-web stdout | 2023-08-08 05:48:16,873 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:16,874 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:48:16,875 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:16,876 [200] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691473696876, 0, 1]) gunicorn-web stdout | 2023-08-08 05:48:16,877 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,878 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,878 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:16,878 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:16,879 [200] [DEBUG] [app] Ending request: urn:request:e4980369-8fb2-4bce-bf6d-255ae2a31418 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:e4980369-8fb2-4bce-bf6d-255ae2a31418', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:16,880 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:16,880 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:16 +0000] "GET /api/v1/user/ HTTP/1.0" 200 673 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:16 +0000] "GET /api/v1/user/ HTTP/2.0" 200 673 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.034 525 0.034) gunicorn-web stdout | 2023-08-08 05:48:17,258 [197] [DEBUG] [app] Starting request: urn:request:f48a66b5-38ff-4906-8f52-23b8de06adad (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:17,258 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,258 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,259 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:17,261 [200] [DEBUG] [app] Starting request: urn:request:7d92c648-1418-4a3a-96c2-4f8efe959885 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:17,261 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,261 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,262 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:17,266 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:17,266 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:17,266 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,267 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:48:17,267 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,267 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,267 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,267 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,268 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:17,268 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:17,268 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,269 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:48:17,269 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,269 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,269 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,269 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,269 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 05:48:17,269 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 0, 17, 269468), datetime.datetime(2023, 8, 8, 5, 48, 17, 269539)]) gunicorn-web stdout | 2023-08-08 05:48:17,271 [200] [DEBUG] [app] Ending request: urn:request:7d92c648-1418-4a3a-96c2-4f8efe959885 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:7d92c648-1418-4a3a-96c2-4f8efe959885', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:17,272 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:17 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 62 0.013) gunicorn-web stdout | 2023-08-08 05:48:17,273 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:17 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:17,274 [197] [DEBUG] [app] Ending request: urn:request:f48a66b5-38ff-4906-8f52-23b8de06adad (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:f48a66b5-38ff-4906-8f52-23b8de06adad', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:17,274 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:17 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.019 119 0.018) gunicorn-web stdout | 2023-08-08 05:48:17,275 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:17 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:17,410 [197] [DEBUG] [app] Starting request: urn:request:dcb27f08-9432-4807-86c6-f761a3c6dab0 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:17,410 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,410 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,411 [200] [DEBUG] [app] Starting request: urn:request:e097e0fb-f666-4524-9bfc-79c8fbe76636 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:17,411 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,411 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,411 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:17,412 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:17,419 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:17,419 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:17,419 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,420 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:17,420 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:17,420 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,420 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:48:17,420 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:17,421 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,421 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,421 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:17,421 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:17,422 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:17,422 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:48:17,423 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:48:17,424 [200] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'whuquay', 101]) gunicorn-web stdout | 2023-08-08 05:48:17,426 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:48:17,428 [197] [DEBUG] [app] Ending request: urn:request:dcb27f08-9432-4807-86c6-f761a3c6dab0 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:dcb27f08-9432-4807-86c6-f761a3c6dab0', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:17,428 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:48:17,428 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:17 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 176 0.020) gunicorn-web stdout | 2023-08-08 05:48:17,429 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:17 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:17,431 [200] [DEBUG] [app] Ending request: urn:request:e097e0fb-f666-4524-9bfc-79c8fbe76636 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:e097e0fb-f666-4524-9bfc-79c8fbe76636', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'whuquay', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:17,431 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:17 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.023 103 0.022) gunicorn-web stdout | 2023-08-08 05:48:17,432 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:17 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" logrotateworker stdout | 2023-08-08 05:48:17,988 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:48:19,169 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:48:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:48:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:29.417660+00:00 (in 9.999485 seconds) gcworker stdout | 2023-08-08 05:48:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:29 UTC)" (scheduled at 2023-08-08 05:48:19.417660+00:00) gcworker stdout | 2023-08-08 05:48:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264099418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:48:19,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:48:19,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:48:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:29 UTC)" executed successfully builder stdout | 2023-08-08 05:48:19,591 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:48:19,599 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:48:19,600 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:48:19,604 [55] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2023-08-08 05:48:19,645 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityworker stdout | 2023-08-08 05:48:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:48:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:26.890741+00:00 (in 7.202907 seconds) securityworker stdout | 2023-08-08 05:48:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:49 UTC)" (scheduled at 2023-08-08 05:48:19.687363+00:00) securityworker stdout | 2023-08-08 05:48:19,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:48:19,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:48:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:48:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:48:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:48:19,700 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:48:19,700 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:49 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:22,179 [197] [DEBUG] [app] Starting request: urn:request:bd4808c8-f526-49fa-b302-877f3bee6738 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:48:22,179 [199] [DEBUG] [app] Starting request: urn:request:1b6a5771-b3c5-4a4c-a904-10d1b767fa29 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:48:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:48:22,184 [203] [DEBUG] [app] Starting request: urn:request:3547b3ae-6e07-49a0-b538-c659216979aa (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:48:22,185 [209] [DEBUG] [app] Starting request: urn:request:fa44619a-41ac-4328-afea-8b8ff696ba4c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:48:22,185 [203] [DEBUG] [app] Ending request: urn:request:3547b3ae-6e07-49a0-b538-c659216979aa (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3547b3ae-6e07-49a0-b538-c659216979aa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:48:22,185 [209] [DEBUG] [app] Ending request: urn:request:fa44619a-41ac-4328-afea-8b8ff696ba4c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fa44619a-41ac-4328-afea-8b8ff696ba4c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:48:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:48:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:48:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:22,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:22,190 [197] [DEBUG] [app] Starting request: urn:request:cce213d5-ee66-44ed-a7d7-715f0ec25334 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:48:22,191 [197] [DEBUG] [app] Ending request: urn:request:cce213d5-ee66-44ed-a7d7-715f0ec25334 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:cce213d5-ee66-44ed-a7d7-715f0ec25334', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:48:22,191 [200] [DEBUG] [app] Starting request: urn:request:b2e22ed9-e535-4624-86f1-9e2dcbf3aa17 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:48:22,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:48:22,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:22,191 [200] [DEBUG] [app] Ending request: urn:request:b2e22ed9-e535-4624-86f1-9e2dcbf3aa17 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b2e22ed9-e535-4624-86f1-9e2dcbf3aa17', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:48:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:48:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:48:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:48:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:48:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:48:22,202 [197] [DEBUG] [app] Ending request: urn:request:bd4808c8-f526-49fa-b302-877f3bee6738 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bd4808c8-f526-49fa-b302-877f3bee6738', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:48:22,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:48:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:48:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 05:48:22,203 [199] [DEBUG] [app] Ending request: urn:request:1b6a5771-b3c5-4a4c-a904-10d1b767fa29 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1b6a5771-b3c5-4a4c-a904-10d1b767fa29', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:48:22,203 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:48:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 05:48:22,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:48:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:48:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:48:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:28.874906+00:00 (in 6.580620 seconds) chunkcleanupworker stdout | 2023-08-08 05:48:22,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:32 UTC)" (scheduled at 2023-08-08 05:48:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:48:22,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:48:22,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 22, 295244), True, datetime.datetime(2023, 8, 8, 5, 48, 22, 295244), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:48:22,335 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:48:22,335 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:48:22,335 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:48:22,407 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:48:24,206 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:48:24,318 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:48:24,539 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:48:24,640 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:48:24,642 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:48:24,646 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:48:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:48:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:35.371682+00:00 (in 9.999445 seconds) notificationworker stdout | 2023-08-08 05:48:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:35 UTC)" (scheduled at 2023-08-08 05:48:25.371682+00:00) notificationworker stdout | 2023-08-08 05:48:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:48:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 25, 372851), True, datetime.datetime(2023, 8, 8, 5, 48, 25, 372851), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:48:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:48:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:48:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:48:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:48:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:33.514094+00:00 (in 7.999239 seconds) exportactionlogsworker stdout | 2023-08-08 05:48:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:25 UTC)" (scheduled at 2023-08-08 05:48:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:48:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:48:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:48:25,649 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:48:26,201 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:48:26,413 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:48:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:48:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:49.687363+00:00 (in 22.796128 seconds) securityworker stdout | 2023-08-08 05:48:26,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:56 UTC)" (scheduled at 2023-08-08 05:48:26.890741+00:00) securityworker stdout | 2023-08-08 05:48:26,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:48:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:48:26,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:48:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:48:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:48:26,894 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:48:26,894 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:48:26,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:48:26,898 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:48:26,904 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:48:26,904 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:48:26,904 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:48:26,905 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:48:26,905 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:48:26,905 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:48:26,905 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:48:26,905 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:48:26,906 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:48:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:27,019 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:48:27,023 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:48:27,025 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:48:27,030 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:48:27,033 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:48:27,637 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:48:27,736 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:48:28,858 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 05:48:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:48:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:32.293757+00:00 (in 3.418333 seconds) chunkcleanupworker stdout | 2023-08-08 05:48:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:28 UTC)" (scheduled at 2023-08-08 05:48:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:48:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:48:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:28 UTC)" executed successfully gcworker stdout | 2023-08-08 05:48:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:48:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:39.417660+00:00 (in 9.999482 seconds) gcworker stdout | 2023-08-08 05:48:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:39 UTC)" (scheduled at 2023-08-08 05:48:29.417660+00:00) gcworker stdout | 2023-08-08 05:48:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264109418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:48:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:48:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:48:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:39 UTC)" executed successfully builder stdout | 2023-08-08 05:48:29,614 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:48:29,623 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:48:29,623 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:48:29,628 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:48:29,650 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:48:29,693 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:48:30,220 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} gunicorn-web stdout | 2023-08-08 05:48:30,322 [197] [DEBUG] [app] Starting request: urn:request:d0a7615a-83ed-4fe4-8477-90736d2982d0 (/api/v1/organization/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:30,323 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,323 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,323 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,330 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:30,330 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:30,330 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,330 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:48:30,331 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,331 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,331 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,331 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,333 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,334 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,337 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['build_test', '5326a5da-c18f-4db1-8581-8bdbb1ad34b6', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,338 [197] [DEBUG] [data.model.user] Email and username are unique! gunicorn-web stdout | 2023-08-08 05:48:30,339 [197] [DEBUG] [peewee] ('INSERT INTO "user" ("uuid", "username", "email", "verified", "organization", "robot", "invoice_email", "invalid_login_attempts", "last_invalid_login", "removed_tag_expiration_s", "enabled", "maximum_queued_builds_count", "creation_date") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "user"."id"', ['37f536ce-08ec-429f-86f1-71fe4f2d8b26', 'build_test', '5326a5da-c18f-4db1-8581-8bdbb1ad34b6', False, False, False, False, 0, datetime.datetime(2023, 8, 8, 5, 48, 30, 338869), 1209600, True, None, datetime.datetime(2023, 8, 8, 5, 48, 30, 338871)]) gunicorn-web stdout | 2023-08-08 05:48:30,341 [197] [DEBUG] [peewee] ('UPDATE "user" SET "uuid" = %s, "username" = %s, "email" = %s, "verified" = %s, "organization" = %s, "robot" = %s, "invoice_email" = %s, "invalid_login_attempts" = %s, "last_invalid_login" = %s, "removed_tag_expiration_s" = %s, "enabled" = %s, "maximum_queued_builds_count" = %s, "creation_date" = %s WHERE ("user"."id" = %s)', ['37f536ce-08ec-429f-86f1-71fe4f2d8b26', 'build_test', '5326a5da-c18f-4db1-8581-8bdbb1ad34b6', False, True, False, False, 0, datetime.datetime(2023, 8, 8, 5, 48, 30, 338869), 1209600, True, None, datetime.datetime(2023, 8, 8, 5, 48, 30, 338871), 2]) gunicorn-web stdout | 2023-08-08 05:48:30,342 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,343 [197] [DEBUG] [peewee] ('INSERT INTO "team" ("name", "organization_id", "role_id", "description") VALUES (%s, %s, %s, %s) RETURNING "team"."id"', ['owners', 2, 1, '']) gunicorn-web stdout | 2023-08-08 05:48:30,345 [197] [DEBUG] [peewee] ('INSERT INTO "teammember" ("user_id", "team_id") VALUES (%s, %s) RETURNING "teammember"."id"', [1, 1]) gunicorn-web stdout | 2023-08-08 05:48:30,349 [197] [DEBUG] [app] Ending request: urn:request:d0a7615a-83ed-4fe4-8477-90736d2982d0 (/api/v1/organization/) {'endpoint': 'api.organizationlist', 'request_id': 'urn:request:d0a7615a-83ed-4fe4-8477-90736d2982d0', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/', 'path': '/api/v1/organization/', 'parameters': {}, 'json_body': {'name': 'build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:30,350 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:30 +0000] "POST /api/v1/organization/ HTTP/2.0" 201 10 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 948 0.029) gunicorn-web stdout | 2023-08-08 05:48:30,350 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:30 +0000] "POST /api/v1/organization/ HTTP/1.0" 201 10 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:30,664 [197] [DEBUG] [app] Starting request: urn:request:386f3350-3136-4d87-9126-99303f966632 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:30,664 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,664 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,665 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,671 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:30,671 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:30,671 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,671 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,671 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,671 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,671 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,672 [197] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 05:48:30,675 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,675 [197] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 05:48:30,677 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:48:30,679 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,680 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:48:30,682 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,684 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691473710683, 0, 1, None]) gunicorn-web stdout | 2023-08-08 05:48:30,687 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,688 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:48:30,690 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,691 [197] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691473710691, 0, 1]) gunicorn-web stdout | 2023-08-08 05:48:30,693 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,693 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,694 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:48:30,696 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,696 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,696 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,696 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:30,697 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,697 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,697 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,697 [200] [DEBUG] [app] Starting request: urn:request:a2aacbe3-600c-4863-aaf3-670203ee1616 (/api/v1/organization/build_test) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:30,697 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,697 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,698 [197] [DEBUG] [app] Ending request: urn:request:386f3350-3136-4d87-9126-99303f966632 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:386f3350-3136-4d87-9126-99303f966632', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:30,698 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,698 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:30 +0000] "GET /api/v1/user/ HTTP/2.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.037 391 0.037) gunicorn-web stdout | 2023-08-08 05:48:30,699 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:30 +0000] "GET /api/v1/user/ HTTP/1.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:30,705 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:30,705 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:30,705 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,706 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,707 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,707 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,707 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,707 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,709 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:48:30,711 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,711 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,712 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:48:30,712 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:30,712 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t2" ON ("t1"."role_id" = "t2"."id") WHERE ("t1"."organization_id" = %s)', [2]) gunicorn-web stdout | 2023-08-08 05:48:30,713 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:48:30,714 [200] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "repositorypermission" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [1]) gunicorn-web stdout | 2023-08-08 05:48:30,716 [200] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "teammember" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [1]) gunicorn-web stdout | 2023-08-08 05:48:30,717 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,717 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,717 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:30,717 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 05:48:30,719 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,720 [200] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691473710720, 0, 2, None]) gunicorn-web stdout | 2023-08-08 05:48:30,724 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,725 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 05:48:30,726 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:30,728 [200] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691473710727, 0, 2]) gunicorn-web stdout | 2023-08-08 05:48:30,730 [200] [DEBUG] [app] Ending request: urn:request:a2aacbe3-600c-4863-aaf3-670203ee1616 (/api/v1/organization/build_test) {'endpoint': 'api.organization', 'request_id': 'urn:request:a2aacbe3-600c-4863-aaf3-670203ee1616', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test', 'path': '/api/v1/organization/build_test', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:30,731 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:30 +0000] "GET /api/v1/organization/build_test HTTP/2.0" 200 770 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.036 139 0.035) gunicorn-web stdout | 2023-08-08 05:48:30,731 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:30 +0000] "GET /api/v1/organization/build_test HTTP/1.0" 200 770 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityscanningnotificationworker stdout | 2023-08-08 05:48:30,793 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-web stdout | 2023-08-08 05:48:31,020 [200] [DEBUG] [app] Starting request: urn:request:bd4ccd59-2256-4679-9ddf-e0773c98b936 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:31,020 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,021 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,021 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,023 [197] [DEBUG] [app] Starting request: urn:request:9d42493f-3e78-45c7-b726-756661ec75eb (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:31,023 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,023 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,024 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,028 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:31,028 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:31,028 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,029 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:48:31,029 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,029 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,029 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,029 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,030 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:31,030 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:31,030 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,031 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:48:31,031 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,031 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,031 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 05:48:31,031 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,031 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,031 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 0, 31, 31342), datetime.datetime(2023, 8, 8, 5, 48, 31, 31400)]) gunicorn-web stdout | 2023-08-08 05:48:31,033 [197] [DEBUG] [app] Ending request: urn:request:9d42493f-3e78-45c7-b726-756661ec75eb (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:9d42493f-3e78-45c7-b726-756661ec75eb', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:31,034 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 62 0.013) gunicorn-web stdout | 2023-08-08 05:48:31,034 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:31,035 [200] [DEBUG] [app] Ending request: urn:request:bd4ccd59-2256-4679-9ddf-e0773c98b936 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:bd4ccd59-2256-4679-9ddf-e0773c98b936', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:31,036 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:31,037 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 54 0.018) namespacegcworker stdout | 2023-08-08 05:48:31,073 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-web stdout | 2023-08-08 05:48:31,200 [200] [DEBUG] [app] Starting request: urn:request:3236312f-c9b5-42af-9882-18f916a13e3e (/api/v1/organization/build_test/quota) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:31,200 [197] [DEBUG] [app] Starting request: urn:request:82ded471-4331-4542-af36-ecc7196d6b42 (/api/v1/organization/build_test/proxycache) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:31,200 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,200 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,200 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,200 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,201 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,201 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,207 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:31,207 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:31,207 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:31,207 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,207 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:31,207 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,208 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,208 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,208 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,208 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,208 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,208 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,209 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:48:31,212 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,212 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,212 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,212 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:31,213 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,214 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 05:48:31,215 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."quota_id", "t1"."quota_type_id", "t1"."percent_of_limit" FROM "quotalimits" AS "t1" INNER JOIN "quotatype" AS "t2" ON ("t1"."quota_type_id" = "t2"."id") WHERE ("t1"."quota_id" = %s)', [None]) gunicorn-web stdout | 2023-08-08 05:48:31,218 [200] [DEBUG] [app] Ending request: urn:request:3236312f-c9b5-42af-9882-18f916a13e3e (/api/v1/organization/build_test/quota) {'endpoint': 'api.organizationquotalist', 'request_id': 'urn:request:3236312f-c9b5-42af-9882-18f916a13e3e', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/quota', 'path': '/api/v1/organization/build_test/quota', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:31,219 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:31,219 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/organization/build_test/quota HTTP/1.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/organization/build_test/quota HTTP/2.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 62 0.021) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:31 +0000] "GET /static/img/creditcards/credit.png HTTP/2.0" 200 1776 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 92 -) gunicorn-web stdout | 2023-08-08 05:48:31,337 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,337 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,340 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:48:31,341 [198] [DEBUG] [app] Starting request: urn:request:c89e2832-a81b-4351-89a9-677143a6c11b (/api/v1/organization/build_test/prototypes) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:31,341 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,341 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,342 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,343 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,343 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,343 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,343 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:31,344 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', True, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,346 [197] [DEBUG] [app] Ending request: urn:request:82ded471-4331-4542-af36-ecc7196d6b42 (/api/v1/organization/build_test/proxycache) {'endpoint': 'api.organizationproxycacheconfig', 'request_id': 'urn:request:82ded471-4331-4542-af36-ecc7196d6b42', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/proxycache', 'path': '/api/v1/organization/build_test/proxycache', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:31,347 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/organization/build_test/proxycache HTTP/2.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.149 66 0.149) gunicorn-web stdout | 2023-08-08 05:48:31,347 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/organization/build_test/proxycache HTTP/1.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:31,348 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:31,349 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:31,349 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,349 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,349 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,349 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,349 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,351 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) expiredappspecifictokenworker stdout | 2023-08-08 05:48:31,352 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-web stdout | 2023-08-08 05:48:31,353 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,354 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,354 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,354 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:31,354 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,356 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" LEFT OUTER JOIN "user" AS "t2" ON ("t2"."id" = "t1"."activating_user_id") LEFT OUTER JOIN "user" AS "t3" ON ("t3"."id" = "t1"."delegate_user_id") LEFT OUTER JOIN "team" AS "t4" ON ("t4"."id" = "t1"."delegate_team_id") LEFT OUTER JOIN "role" AS "t5" ON ("t5"."id" = "t1"."role_id") WHERE ("t1"."org_id" = %s)', [2]) gunicorn-web stdout | 2023-08-08 05:48:31,359 [198] [DEBUG] [app] Ending request: urn:request:c89e2832-a81b-4351-89a9-677143a6c11b (/api/v1/organization/build_test/prototypes) {'endpoint': 'api.permissionprototypelist', 'request_id': 'urn:request:c89e2832-a81b-4351-89a9-677143a6c11b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/prototypes', 'path': '/api/v1/organization/build_test/prototypes', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:31,360 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/organization/build_test/prototypes HTTP/2.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 410 0.021) gunicorn-web stdout | 2023-08-08 05:48:31,360 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/organization/build_test/prototypes HTTP/1.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:31,376 [200] [DEBUG] [app] Starting request: urn:request:92e3d891-1b6e-4996-babe-6ff96811debf (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:31,376 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,376 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,377 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,383 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:31,383 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:31,383 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,384 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:31,384 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,384 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,384 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:31,384 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:31,385 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,386 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:31,388 [200] [DEBUG] [peewee] ('(SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t1"."visibility_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") WHERE ((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'build_test', 1, 3, 1, 'build_test', 101]) gunicorn-web stdout | 2023-08-08 05:48:31,392 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:48:31,394 [200] [DEBUG] [app] Ending request: urn:request:92e3d891-1b6e-4996-babe-6ff96811debf (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:92e3d891-1b6e-4996-babe-6ff96811debf', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'true', 'quota': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:31,394 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:31,395 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:31 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 106 0.020) chunkcleanupworker stdout | 2023-08-08 05:48:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:48:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:42.293757+00:00 (in 9.999483 seconds) chunkcleanupworker stdout | 2023-08-08 05:48:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:42 UTC)" (scheduled at 2023-08-08 05:48:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:48:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:48:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 32, 294769), True, datetime.datetime(2023, 8, 8, 5, 48, 32, 294769), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:48:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:48:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:48:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:48:32,823 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:48:32,826 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:48:32,829 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:48:32,834 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:48:32,838 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:48:32,840 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:48:32,844 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:48:32,847 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:48:32,849 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} exportactionlogsworker stdout | 2023-08-08 05:48:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:48:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:47.398551+00:00 (in 13.883839 seconds) exportactionlogsworker stdout | 2023-08-08 05:48:33,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:53:33 UTC)" (scheduled at 2023-08-08 05:48:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 05:48:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 48, 33, 515170), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:48:33,522 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 48, 33, 515170), True, datetime.datetime(2023, 8, 8, 5, 48, 33, 515170), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:48:33,524 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 48, 33, 515170), True, datetime.datetime(2023, 8, 8, 5, 48, 33, 515170), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 5, 48, 33, 515170), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:48:33,525 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:48:33,526 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:53:33 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:33 +0000] "GET /static/fonts/glyphicons-halflings-regular.woff2 HTTP/2.0" 200 18028 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/static/ldn/bootstrap.min.css-13f668494557.css" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 228 -) notificationworker stdout | 2023-08-08 05:48:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:48:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:45.371682+00:00 (in 9.999437 seconds) notificationworker stdout | 2023-08-08 05:48:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:45 UTC)" (scheduled at 2023-08-08 05:48:35.371682+00:00) notificationworker stdout | 2023-08-08 05:48:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:48:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 35, 372810), True, datetime.datetime(2023, 8, 8, 5, 48, 35, 372810), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:48:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:48:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:48:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:48:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:48:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:54.393627+00:00 (in 17.813232 seconds) namespacegcworker stdout | 2023-08-08 05:48:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:36 UTC)" (scheduled at 2023-08-08 05:48:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:48:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:48:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:37,179 [197] [DEBUG] [app] Starting request: urn:request:1426555c-1aba-4477-8da6-96f2d890a71e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:48:37,179 [199] [DEBUG] [app] Starting request: urn:request:d2d0b5ac-73a3-44fa-8eac-f1fb40e4155f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:48:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:37,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:48:37,184 [203] [DEBUG] [app] Starting request: urn:request:0d5ef7b0-8d3f-4438-86e6-5d97bf7d024c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:48:37,185 [210] [DEBUG] [app] Starting request: urn:request:345ae9a1-4bde-4dbd-b03c-418e1797736e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:48:37,185 [203] [DEBUG] [app] Ending request: urn:request:0d5ef7b0-8d3f-4438-86e6-5d97bf7d024c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0d5ef7b0-8d3f-4438-86e6-5d97bf7d024c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:48:37,185 [210] [DEBUG] [app] Ending request: urn:request:345ae9a1-4bde-4dbd-b03c-418e1797736e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:345ae9a1-4bde-4dbd-b03c-418e1797736e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:48:37,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:48:37,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 05:48:37,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:37,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:37,191 [197] [DEBUG] [app] Starting request: urn:request:c5f1a6ec-9b3b-4eff-acbf-1ca2126efb68 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:48:37,191 [197] [DEBUG] [app] Ending request: urn:request:c5f1a6ec-9b3b-4eff-acbf-1ca2126efb68 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c5f1a6ec-9b3b-4eff-acbf-1ca2126efb68', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:48:37,191 [200] [DEBUG] [app] Starting request: urn:request:33217a14-348c-45c4-826e-16ba4ec206cd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:48:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:37,192 [200] [DEBUG] [app] Ending request: urn:request:33217a14-348c-45c4-826e-16ba4ec206cd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:33217a14-348c-45c4-826e-16ba4ec206cd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:48:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:48:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:37,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:37,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:48:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:48:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:07.197412+00:00 (in 29.999517 seconds) gunicorn-web stdout | 2023-08-08 05:48:37,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:48:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:07 UTC)" (scheduled at 2023-08-08 05:48:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:48:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 48, 37, 198329), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,200 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:48:37,205 [199] [DEBUG] [app] Ending request: urn:request:d2d0b5ac-73a3-44fa-8eac-f1fb40e4155f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d2d0b5ac-73a3-44fa-8eac-f1fb40e4155f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:48:37,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:37,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:48:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:48:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) gunicorn-web stdout | 2023-08-08 05:48:37,206 [197] [DEBUG] [app] Ending request: urn:request:1426555c-1aba-4477-8da6-96f2d890a71e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1426555c-1aba-4477-8da6-96f2d890a71e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:48:37,206 [197] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:48:37,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:48:37,206 [56] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:48:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.029) gunicorn-web stdout | 2023-08-08 05:48:37,207 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:48:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:48:37,207 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:37,822 [197] [DEBUG] [app] Starting request: urn:request:8dca6272-2ecf-403b-b838-f1b747c6f349 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:37,823 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:37,823 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:37,824 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,831 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:37,831 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:37,831 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:37,833 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:37,834 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:37,834 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:37,834 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:37,836 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:48:37,839 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:37,839 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:48:37,839 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:48:37,839 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:48:37,841 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,844 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,846 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,848 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:48:37,849 [197] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [2, 'gitlab', 2, '', '0fe857a5-447d-43a8-9f00-b707338bb0d3', 1, False, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,852 [197] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [1, 0, datetime.date(2023, 8, 7)]) gunicorn-web stdout | 2023-08-08 05:48:37,853 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,855 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,856 [197] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [1, 1, 1]) gunicorn-web stdout | 2023-08-08 05:48:37,860 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_user_id" IS %s))', [2, 1, None, None]) gunicorn-web stdout | 2023-08-08 05:48:37,862 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_team_id" IS %s))', [2, 1, None, None]) gunicorn-web stdout | 2023-08-08 05:48:37,863 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,865 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:37,866 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "logentrykind" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:48:37,869 [197] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [14, 2, 1, 1, datetime.datetime(2023, 8, 8, 5, 48, 37, 865308), '10.130.2.1', '{"repo": "gitlab", "namespace": "build_test"}']) gunicorn-web stdout | 2023-08-08 05:48:37,873 [197] [DEBUG] [app] Ending request: urn:request:8dca6272-2ecf-403b-b838-f1b747c6f349 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:8dca6272-2ecf-403b-b838-f1b747c6f349', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository', 'path': '/api/v1/repository', 'parameters': {}, 'json_body': {'namespace': 'build_test', 'repository': 'gitlab', 'visibility': 'private', 'description': '', 'repo_kind': 'image'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:37,874 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:37,874 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:37 +0000] "POST /api/v1/repository HTTP/1.0" 201 63 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:37 +0000] "POST /api/v1/repository HTTP/2.0" 201 63 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.054 247 0.053) gunicorn-web stdout | 2023-08-08 05:48:38,282 [198] [DEBUG] [app] Starting request: urn:request:7798aed6-c5ac-4a71-b0fb-59774dbed439 (/api/v1/repository/build_test/gitlab/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:38,283 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,283 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,284 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,284 [197] [DEBUG] [app] Starting request: urn:request:2238769e-a4ea-41c0-b5c8-06c25a1f5a2a (/api/v1/repository/build_test/gitlab) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:38,284 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,284 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,285 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,291 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:38,291 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:38,291 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,291 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:48:38,292 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,292 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:38,292 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:38,292 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:38,292 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:38,292 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:38,292 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,292 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:48:38,292 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,292 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:38,292 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:38,292 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:38,295 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:48:38,295 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:48:38,301 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:48:38,301 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:48:38,302 [197] [DEBUG] [endpoints.api.repository] Get repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:48:38,302 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,303 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,305 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [1, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,305 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,307 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,307 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:48:38,308 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,308 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [1, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:48:38,309 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,310 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:38,310 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [1, None, 1691473718309, False, 101, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,311 [197] [DEBUG] [app] Ending request: urn:request:2238769e-a4ea-41c0-b5c8-06c25a1f5a2a (/api/v1/repository/build_test/gitlab) {'endpoint': 'api.repository', 'request_id': 'urn:request:2238769e-a4ea-41c0-b5c8-06c25a1f5a2a', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/gitlab', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:38,311 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:38,312 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:38 +0000] "GET /api/v1/repository/build_test/gitlab?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:38 +0000] "GET /api/v1/repository/build_test/gitlab?includeStats=true&includeTags=false HTTP/2.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 338 0.030) gunicorn-web stdout | 2023-08-08 05:48:38,313 [198] [DEBUG] [app] Ending request: urn:request:7798aed6-c5ac-4a71-b0fb-59774dbed439 (/api/v1/repository/build_test/gitlab/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:7798aed6-c5ac-4a71-b0fb-59774dbed439', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/gitlab/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:38,313 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:38 +0000] "GET /api/v1/repository/build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.033 536 0.033) gunicorn-web stdout | 2023-08-08 05:48:38,314 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:38 +0000] "GET /api/v1/repository/build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:38,985 [200] [DEBUG] [app] Starting request: urn:request:804ad5c7-a30a-4d2b-86b5-df59c5b80f70 (/repository/build_test/gitlab/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:38,987 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,997 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:38,999 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:39,002 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['gitlab', 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:39,008 [200] [DEBUG] [app] Ending request: urn:request:804ad5c7-a30a-4d2b-86b5-df59c5b80f70 (/repository/build_test/gitlab/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:804ad5c7-a30a-4d2b-86b5-df59c5b80f70', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab/status?token=0fe857a5-447d-43a8-9f00-b707338bb0d3', 'path': '/repository/build_test/gitlab/status', 'parameters': {'token': '0fe857a5-447d-43a8-9f00-b707338bb0d3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:39,009 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:39,009 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:39 +0000] "GET /repository/build_test/gitlab/status?token=0fe857a5-447d-43a8-9f00-b707338bb0d3 HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:39 +0000] "GET /repository/build_test/gitlab/status?token=0fe857a5-447d-43a8-9f00-b707338bb0d3 HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 436 0.026) gunicorn-web stdout | 2023-08-08 05:48:39,149 [197] [DEBUG] [app] Starting request: urn:request:53779245-ed94-4d32-967e-fd7c986f14ec (/api/v1/repository/build_test/gitlab/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:39,149 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:39,149 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:39,150 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:39,156 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:39,157 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:39,157 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:39,157 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:48:39,157 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:39,157 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:39,157 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:39,157 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:39,161 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:48:39,167 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:48:39,168 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:39,173 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['gitlab', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:48:39,176 [197] [DEBUG] [app] Ending request: urn:request:53779245-ed94-4d32-967e-fd7c986f14ec (/api/v1/repository/build_test/gitlab/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:53779245-ed94-4d32-967e-fd7c986f14ec', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/build/?limit=3', 'path': '/api/v1/repository/build_test/gitlab/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:39,177 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:39,178 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:39 +0000] "GET /api/v1/repository/build_test/gitlab/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:39 +0000] "GET /api/v1/repository/build_test/gitlab/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 79 0.030) gcworker stdout | 2023-08-08 05:48:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:48:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:49.417660+00:00 (in 9.999460 seconds) gcworker stdout | 2023-08-08 05:48:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:49 UTC)" (scheduled at 2023-08-08 05:48:39.417660+00:00) gcworker stdout | 2023-08-08 05:48:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264119418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:48:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:48:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:48:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:49 UTC)" executed successfully builder stdout | 2023-08-08 05:48:39,638 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:48:39,648 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:48:39,648 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:48:39,652 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:40,437 [199] [DEBUG] [app] Starting request: urn:request:af05d83d-e2f3-42b2-8724-fd1ea4ef3ddd (/api/v1/repository/build_test/gitlab/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:40,438 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:40,438 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:40,439 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:40,442 [197] [DEBUG] [app] Starting request: urn:request:63c923b0-d4ff-4f3e-b960-41fa2afa72dd (/api/v1/repository/build_test/gitlab/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:48:40,442 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:40,442 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:40,443 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:40,445 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:40,445 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:40,446 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:40,446 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:48:40,446 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:40,446 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:40,446 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:40,446 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:40,449 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:48:40,450 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:48:40,450 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:48:40,450 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:40,450 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:48:40,450 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:48:40,450 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:40,450 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:40,450 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:48:40,453 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:48:40,456 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:48:40,457 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:40,459 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:48:40,460 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:48:40,460 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'gitlab']) gunicorn-web stdout | 2023-08-08 05:48:40,462 [199] [DEBUG] [app] Ending request: urn:request:af05d83d-e2f3-42b2-8724-fd1ea4ef3ddd (/api/v1/repository/build_test/gitlab/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:af05d83d-e2f3-42b2-8724-fd1ea4ef3ddd', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/trigger/', 'path': '/api/v1/repository/build_test/gitlab/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:40,463 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:40 +0000] "GET /api/v1/repository/build_test/gitlab/trigger/ HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 590 0.028) gunicorn-web stdout | 2023-08-08 05:48:40,464 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['gitlab', 'build_test', 10]) gunicorn-web stdout | 2023-08-08 05:48:40,464 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:40 +0000] "GET /api/v1/repository/build_test/gitlab/trigger/ HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:48:40,467 [197] [DEBUG] [app] Ending request: urn:request:63c923b0-d4ff-4f3e-b960-41fa2afa72dd (/api/v1/repository/build_test/gitlab/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:63c923b0-d4ff-4f3e-b960-41fa2afa72dd', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/build/?limit=10', 'path': '/api/v1/repository/build_test/gitlab/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:48:40,468 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:48:40 +0000] "GET /api/v1/repository/build_test/gitlab/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 73 0.028) gunicorn-web stdout | 2023-08-08 05:48:40,468 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:48:40 +0000] "GET /api/v1/repository/build_test/gitlab/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" chunkcleanupworker stdout | 2023-08-08 05:48:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:48:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:52.293757+00:00 (in 9.999480 seconds) chunkcleanupworker stdout | 2023-08-08 05:48:42,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:52 UTC)" (scheduled at 2023-08-08 05:48:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:48:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:48:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 42, 295503), True, datetime.datetime(2023, 8, 8, 5, 48, 42, 295503), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:48:42,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:48:42,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:48:42,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:48:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:48:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:49.178248+00:00 (in 3.805982 seconds) notificationworker stdout | 2023-08-08 05:48:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:55 UTC)" (scheduled at 2023-08-08 05:48:45.371682+00:00) notificationworker stdout | 2023-08-08 05:48:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:48:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 45, 372848), True, datetime.datetime(2023, 8, 8, 5, 48, 45, 372848), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:48:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:48:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:48:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:48:46,820 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:48:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:48:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:25.514341+00:00 (in 38.115245 seconds) exportactionlogsworker stdout | 2023-08-08 05:48:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:47 UTC)" (scheduled at 2023-08-08 05:48:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:48:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:48:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 47, 399653), True, datetime.datetime(2023, 8, 8, 5, 48, 47, 399653), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:48:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:48:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:48:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:48:48,025 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} notificationworker stdout | 2023-08-08 05:48:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:48:49,178 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:55.371682+00:00 (in 6.192860 seconds) notificationworker stdout | 2023-08-08 05:48:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:53:49 UTC)" (scheduled at 2023-08-08 05:48:49.178248+00:00) notificationworker stdout | 2023-08-08 05:48:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 48, 49, 179277), 'notification/%']) notificationworker stdout | 2023-08-08 05:48:49,187 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 48, 49, 179277), True, datetime.datetime(2023, 8, 8, 5, 48, 49, 179277), 0, 'notification/%']) notificationworker stdout | 2023-08-08 05:48:49,189 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 48, 49, 179277), True, datetime.datetime(2023, 8, 8, 5, 48, 49, 179277), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 5, 48, 49, 179277), 'notification/%']) notificationworker stdout | 2023-08-08 05:48:49,190 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:48:49,190 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:53:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:48:49,198 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:48:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:48:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:59.417660+00:00 (in 9.999509 seconds) gcworker stdout | 2023-08-08 05:48:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:59 UTC)" (scheduled at 2023-08-08 05:48:49.417660+00:00) gcworker stdout | 2023-08-08 05:48:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264129418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:48:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:48:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:48:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:48:59 UTC)" executed successfully builder stdout | 2023-08-08 05:48:49,663 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:48:49,672 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:48:49,672 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:48:49,677 [55] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2023-08-08 05:48:49,677 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityworker stdout | 2023-08-08 05:48:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:48:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:48:56.890741+00:00 (in 7.202899 seconds) securityworker stdout | 2023-08-08 05:48:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:19 UTC)" (scheduled at 2023-08-08 05:48:49.687363+00:00) securityworker stdout | 2023-08-08 05:48:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:48:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:48:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:48:49,691 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:48:49,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:48:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:48:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:19 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:52,179 [197] [DEBUG] [app] Starting request: urn:request:6a65f06d-0ba5-4cea-8c7f-d6dfc96ad5f0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:48:52,179 [199] [DEBUG] [app] Starting request: urn:request:60425c01-9267-4f56-b122-ba0bae66a2d0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:48:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:48:52,185 [209] [DEBUG] [app] Starting request: urn:request:49903c37-fb00-4818-be2c-82e3ab854c16 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:48:52,185 [209] [DEBUG] [app] Ending request: urn:request:49903c37-fb00-4818-be2c-82e3ab854c16 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:49903c37-fb00-4818-be2c-82e3ab854c16', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:48:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:48:52,186 [203] [DEBUG] [app] Starting request: urn:request:2c56bacf-9d74-4a3e-bf1c-13402b2ecfd4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:48:52,187 [203] [DEBUG] [app] Ending request: urn:request:2c56bacf-9d74-4a3e-bf1c-13402b2ecfd4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2c56bacf-9d74-4a3e-bf1c-13402b2ecfd4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:48:52,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:48:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:52,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:52,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:48:52,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:52,192 [200] [DEBUG] [app] Starting request: urn:request:2b3e92ba-ab99-4dc0-ad36-645d59a8ab55 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:48:52,193 [200] [DEBUG] [app] Ending request: urn:request:2b3e92ba-ab99-4dc0-ad36-645d59a8ab55 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2b3e92ba-ab99-4dc0-ad36-645d59a8ab55', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:48:52,193 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:48:52,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:52,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:48:52,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:48:52,194 [198] [DEBUG] [app] Starting request: urn:request:b25f9e22-2747-465a-aa38-f67910120c03 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:48:52,195 [198] [DEBUG] [app] Ending request: urn:request:b25f9e22-2747-465a-aa38-f67910120c03 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b25f9e22-2747-465a-aa38-f67910120c03', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:48:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:48:52,195 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:48:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:48:52,195 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:48:52,196 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:48:52,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:48:52,201 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:48:52,204 [199] [DEBUG] [app] Ending request: urn:request:60425c01-9267-4f56-b122-ba0bae66a2d0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:60425c01-9267-4f56-b122-ba0bae66a2d0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:48:52,204 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:48:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:48:52,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:48:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:48:52,206 [197] [DEBUG] [app] Ending request: urn:request:6a65f06d-0ba5-4cea-8c7f-d6dfc96ad5f0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6a65f06d-0ba5-4cea-8c7f-d6dfc96ad5f0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:48:52,206 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:48:52,207 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:48:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:48:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.029) chunkcleanupworker stdout | 2023-08-08 05:48:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:48:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:02.293757+00:00 (in 9.999468 seconds) chunkcleanupworker stdout | 2023-08-08 05:48:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:02 UTC)" (scheduled at 2023-08-08 05:48:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:48:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:48:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 52, 294771), True, datetime.datetime(2023, 8, 8, 5, 48, 52, 294771), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:48:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:48:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:48:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:48:52,446 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:48:54,231 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityscanningnotificationworker stdout | 2023-08-08 05:48:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:48:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:13.097375+00:00 (in 18.812751 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:48:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:54 UTC)" (scheduled at 2023-08-08 05:48:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:48:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:48:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:54 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:48:54,334 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} namespacegcworker stdout | 2023-08-08 05:48:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:48:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:36.579841+00:00 (in 42.185651 seconds) namespacegcworker stdout | 2023-08-08 05:48:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:54 UTC)" (scheduled at 2023-08-08 05:48:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:48:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:48:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 54, 394773), True, datetime.datetime(2023, 8, 8, 5, 48, 54, 394773), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:48:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:48:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:48:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:49:54 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:48:54,576 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:48:54,648 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:48:54,651 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:48:54,653 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:48:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:48:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:02.178530+00:00 (in 6.806336 seconds) notificationworker stdout | 2023-08-08 05:48:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:05 UTC)" (scheduled at 2023-08-08 05:48:55.371682+00:00) notificationworker stdout | 2023-08-08 05:48:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:48:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 48, 55, 372732), True, datetime.datetime(2023, 8, 8, 5, 48, 55, 372732), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:48:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:48:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:48:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:48:55,686 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:48:56,222 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:48:56,445 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:48:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:48:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:19.687363+00:00 (in 22.796113 seconds) securityworker stdout | 2023-08-08 05:48:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:26 UTC)" (scheduled at 2023-08-08 05:48:56.890741+00:00) securityworker stdout | 2023-08-08 05:48:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:48:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:48:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:48:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:48:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:48:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:48:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:48:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:48:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:48:56,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:48:56,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:48:56,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:48:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:48:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:48:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:48:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:48:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:48:56,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:48:57,030 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:48:57,035 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:48:57,038 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:48:57,042 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:48:57,045 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:48:57,650 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:48:57,745 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:48:58,895 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:48:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:48:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:09.417660+00:00 (in 9.999498 seconds) gcworker stdout | 2023-08-08 05:48:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:09 UTC)" (scheduled at 2023-08-08 05:48:59.417660+00:00) gcworker stdout | 2023-08-08 05:48:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264139418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:48:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:48:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:48:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:48:59,681 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} builder stdout | 2023-08-08 05:48:59,686 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:48:59,695 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:48:59,695 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:48:59,699 [55] [DEBUG] [data.database] Disconnecting from database. servicekey stdout | 2023-08-08 05:48:59,730 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:49:00,233 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:49:00,831 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:49:01,097 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:49:01,389 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:49:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:49:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:05.371682+00:00 (in 3.192553 seconds) notificationworker stdout | 2023-08-08 05:49:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:02 UTC)" (scheduled at 2023-08-08 05:49:02.178530+00:00) notificationworker stdout | 2023-08-08 05:49:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:49:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:49:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:49:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:12.293757+00:00 (in 9.999427 seconds) chunkcleanupworker stdout | 2023-08-08 05:49:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:12 UTC)" (scheduled at 2023-08-08 05:49:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:49:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:49:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 2, 294771), True, datetime.datetime(2023, 8, 8, 5, 49, 2, 294771), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:49:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:49:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:49:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:49:02,834 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:49:02,837 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:49:02,840 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:49:02,846 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:49:02,848 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:49:02,851 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:49:02,855 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:49:02,858 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:49:02,860 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:49:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:49:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:09.793658+00:00 (in 5.605694 seconds) repositorygcworker stdout | 2023-08-08 05:49:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:04 UTC)" (scheduled at 2023-08-08 05:49:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:49:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:49:04,189 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:49:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:49:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:15.371682+00:00 (in 9.999487 seconds) notificationworker stdout | 2023-08-08 05:49:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:15 UTC)" (scheduled at 2023-08-08 05:49:05.371682+00:00) notificationworker stdout | 2023-08-08 05:49:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:49:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 5, 372754), True, datetime.datetime(2023, 8, 8, 5, 49, 5, 372754), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:49:05,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:49:05,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:49:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:49:07,179 [197] [DEBUG] [app] Starting request: urn:request:73faecdf-8bf2-4042-9e67-9105ecac7562 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:49:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:07,181 [200] [DEBUG] [app] Starting request: urn:request:2a911604-3b4e-4a8c-b3d6-a5256373173c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:49:07,183 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:49:07,185 [206] [DEBUG] [app] Starting request: urn:request:0e3c34e5-f981-4048-924e-b669e801aff8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:49:07,185 [206] [DEBUG] [app] Ending request: urn:request:0e3c34e5-f981-4048-924e-b669e801aff8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0e3c34e5-f981-4048-924e-b669e801aff8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:49:07,186 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:49:07,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:49:07,187 [209] [DEBUG] [app] Starting request: urn:request:0178682f-b8c6-4373-8ab1-5eef5c0c728a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:49:07,187 [209] [DEBUG] [app] Ending request: urn:request:0178682f-b8c6-4373-8ab1-5eef5c0c728a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0178682f-b8c6-4373-8ab1-5eef5c0c728a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:49:07,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 05:49:07,188 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:07,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:07,192 [197] [DEBUG] [app] Starting request: urn:request:a545f9c3-4e75-4f3c-a88c-03e3e0493abd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:07,192 [197] [DEBUG] [app] Ending request: urn:request:a545f9c3-4e75-4f3c-a88c-03e3e0493abd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a545f9c3-4e75-4f3c-a88c-03e3e0493abd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:49:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:07,192 [199] [DEBUG] [app] Starting request: urn:request:c6974e4e-f28d-4f59-b867-77201dde1f12 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:07,193 [199] [DEBUG] [app] Ending request: urn:request:c6974e4e-f28d-4f59-b867-77201dde1f12 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c6974e4e-f28d-4f59-b867-77201dde1f12', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:49:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:49:07,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:49:07,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:49:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:49:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:37.197412+00:00 (in 29.999527 seconds) buildlogsarchiver stdout | 2023-08-08 05:49:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:37 UTC)" (scheduled at 2023-08-08 05:49:07.197412+00:00) gunicorn-web stdout | 2023-08-08 05:49:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:49:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:49:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 49, 7, 198314), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:49:07,204 [200] [DEBUG] [app] Ending request: urn:request:2a911604-3b4e-4a8c-b3d6-a5256373173c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2a911604-3b4e-4a8c-b3d6-a5256373173c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:49:07,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:49:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:49:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:49:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 05:49:07,205 [197] [DEBUG] [app] Ending request: urn:request:73faecdf-8bf2-4042-9e67-9105ecac7562 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:73faecdf-8bf2-4042-9e67-9105ecac7562', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:49:07,205 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:49:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:49:07,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:49:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:49:07,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:49:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:49:07,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:37 UTC)" executed successfully gcworker stdout | 2023-08-08 05:49:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:49:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:19.417660+00:00 (in 9.999504 seconds) gcworker stdout | 2023-08-08 05:49:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:19 UTC)" (scheduled at 2023-08-08 05:49:09.417660+00:00) gcworker stdout | 2023-08-08 05:49:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264149418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:49:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:49:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:49:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:19 UTC)" executed successfully builder stdout | 2023-08-08 05:49:09,709 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:49:09,718 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:49:09,718 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:49:09,723 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:49:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:49:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:19.187230+00:00 (in 9.393072 seconds) repositorygcworker stdout | 2023-08-08 05:49:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:09 UTC)" (scheduled at 2023-08-08 05:49:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:49:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:49:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 9, 794764), True, datetime.datetime(2023, 8, 8, 5, 49, 9, 794764), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:49:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:49:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:49:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:49:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:49:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:22.293757+00:00 (in 9.999447 seconds) chunkcleanupworker stdout | 2023-08-08 05:49:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:22 UTC)" (scheduled at 2023-08-08 05:49:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:49:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:49:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 12, 294719), True, datetime.datetime(2023, 8, 8, 5, 49, 12, 294719), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:49:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:49:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:49:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:49:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:49:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:54.284093+00:00 (in 41.186187 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:49:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:13 UTC)" (scheduled at 2023-08-08 05:49:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:49:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:49:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 13, 98270), True, datetime.datetime(2023, 8, 8, 5, 49, 13, 98270), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:49:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:49:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:49:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:49:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:49:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:25.371682+00:00 (in 9.999438 seconds) notificationworker stdout | 2023-08-08 05:49:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:25 UTC)" (scheduled at 2023-08-08 05:49:15.371682+00:00) notificationworker stdout | 2023-08-08 05:49:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:49:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 15, 372868), True, datetime.datetime(2023, 8, 8, 5, 49, 15, 372868), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:49:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:49:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:49:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:49:16,858 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:49:18,046 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} repositorygcworker stdout | 2023-08-08 05:49:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:49:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:04.187429+00:00 (in 44.999668 seconds) repositorygcworker stdout | 2023-08-08 05:49:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:54:19 UTC)" (scheduled at 2023-08-08 05:49:19.187230+00:00) repositorygcworker stdout | 2023-08-08 05:49:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 49, 19, 188121), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:49:19,195 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 49, 19, 188121), True, datetime.datetime(2023, 8, 8, 5, 49, 19, 188121), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:49:19,197 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 49, 19, 188121), True, datetime.datetime(2023, 8, 8, 5, 49, 19, 188121), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 5, 49, 19, 188121), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:49:19,198 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:49:19,198 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:54:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:49:19,234 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:49:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:49:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:29.417660+00:00 (in 9.999517 seconds) gcworker stdout | 2023-08-08 05:49:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:29 UTC)" (scheduled at 2023-08-08 05:49:19.417660+00:00) gcworker stdout | 2023-08-08 05:49:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264159418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:49:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:49:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:49:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:29 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:49:19,683 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityworker stdout | 2023-08-08 05:49:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:49:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:26.890741+00:00 (in 7.202883 seconds) securityworker stdout | 2023-08-08 05:49:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:49 UTC)" (scheduled at 2023-08-08 05:49:19.687363+00:00) securityworker stdout | 2023-08-08 05:49:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:49:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:49:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:49:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:49:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:49:19,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:49:19,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:49 UTC)" executed successfully builder stdout | 2023-08-08 05:49:19,723 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:49:19,732 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:49:19,732 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:49:19,737 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:49:22,180 [200] [DEBUG] [app] Starting request: urn:request:5b2fb948-51ed-4146-aa5f-eb6ed75d5c25 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:49:22,180 [199] [DEBUG] [app] Starting request: urn:request:d6e65d50-351c-457a-843d-ffe1b5e2125f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:49:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:22,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:22,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:49:22,185 [203] [DEBUG] [app] Starting request: urn:request:c9d902c1-b5d0-4178-b3a2-07adacb2d7b3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:49:22,186 [203] [DEBUG] [app] Ending request: urn:request:c9d902c1-b5d0-4178-b3a2-07adacb2d7b3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c9d902c1-b5d0-4178-b3a2-07adacb2d7b3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:49:22,186 [209] [DEBUG] [app] Starting request: urn:request:542979fd-7b06-4783-a3e4-ca9d0e06016b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:49:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:49:22,186 [209] [DEBUG] [app] Ending request: urn:request:542979fd-7b06-4783-a3e4-ca9d0e06016b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:542979fd-7b06-4783-a3e4-ca9d0e06016b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:49:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:49:22,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:49:22,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:22,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:22,192 [197] [DEBUG] [app] Starting request: urn:request:d93a3320-2e5f-46da-be50-1f89a8caee34 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:22,192 [200] [DEBUG] [app] Starting request: urn:request:39f66998-bdda-488c-9a19-195f9a611cfd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:22,192 [197] [DEBUG] [app] Ending request: urn:request:d93a3320-2e5f-46da-be50-1f89a8caee34 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d93a3320-2e5f-46da-be50-1f89a8caee34', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:49:22,193 [200] [DEBUG] [app] Ending request: urn:request:39f66998-bdda-488c-9a19-195f9a611cfd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:39f66998-bdda-488c-9a19-195f9a611cfd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:49:22,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 05:49:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:22,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:49:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:49:22,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:49:22,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:49:22,204 [200] [DEBUG] [app] Ending request: urn:request:5b2fb948-51ed-4146-aa5f-eb6ed75d5c25 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5b2fb948-51ed-4146-aa5f-eb6ed75d5c25', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:49:22,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:49:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:49:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:49:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:49:22,205 [199] [DEBUG] [app] Ending request: urn:request:d6e65d50-351c-457a-843d-ffe1b5e2125f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d6e65d50-351c-457a-843d-ffe1b5e2125f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:49:22,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:49:22,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:49:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:49:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 05:49:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:49:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:28.874906+00:00 (in 6.580644 seconds) chunkcleanupworker stdout | 2023-08-08 05:49:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:32 UTC)" (scheduled at 2023-08-08 05:49:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:49:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:49:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 22, 294637), True, datetime.datetime(2023, 8, 8, 5, 49, 22, 294637), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:49:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:49:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:49:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:49:22,465 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:49:24,266 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:49:24,348 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:49:24,612 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:49:24,656 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:49:24,659 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:49:24,662 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:49:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:49:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:35.371682+00:00 (in 9.999232 seconds) notificationworker stdout | 2023-08-08 05:49:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:35 UTC)" (scheduled at 2023-08-08 05:49:25.371682+00:00) notificationworker stdout | 2023-08-08 05:49:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:49:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 25, 372928), True, datetime.datetime(2023, 8, 8, 5, 49, 25, 372928), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:49:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:49:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:49:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:49:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:49:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:47.398551+00:00 (in 21.883629 seconds) exportactionlogsworker stdout | 2023-08-08 05:49:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:25 UTC)" (scheduled at 2023-08-08 05:49:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:49:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:49:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:49:25,723 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:49:26,259 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:49:26,483 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:49:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:49:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:49.687363+00:00 (in 22.796052 seconds) securityworker stdout | 2023-08-08 05:49:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:56 UTC)" (scheduled at 2023-08-08 05:49:26.890741+00:00) securityworker stdout | 2023-08-08 05:49:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:49:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:49:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:49:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:49:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:49:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:49:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:49:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:49:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:49:26,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:49:26,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:49:26,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:49:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:49:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:49:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:49:26,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:49:26,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:49:26,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:49:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:49:27,041 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:49:27,045 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:49:27,049 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:49:27,052 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:49:27,055 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:49:27,687 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:49:27,756 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:49:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:49:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:32.293757+00:00 (in 3.418360 seconds) chunkcleanupworker stdout | 2023-08-08 05:49:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:28 UTC)" (scheduled at 2023-08-08 05:49:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:49:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:49:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:49:28,905 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:49:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:49:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:39.417660+00:00 (in 9.999498 seconds) gcworker stdout | 2023-08-08 05:49:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:39 UTC)" (scheduled at 2023-08-08 05:49:29.417660+00:00) gcworker stdout | 2023-08-08 05:49:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264169418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:49:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:49:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:49:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:49:29,720 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} builder stdout | 2023-08-08 05:49:29,738 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:49:29,747 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:49:29,747 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:49:29,753 [55] [DEBUG] [data.database] Disconnecting from database. servicekey stdout | 2023-08-08 05:49:29,767 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:49:30,270 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:49:30,869 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:49:31,123 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:49:31,396 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:49:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:49:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:42.293757+00:00 (in 9.999468 seconds) chunkcleanupworker stdout | 2023-08-08 05:49:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:42 UTC)" (scheduled at 2023-08-08 05:49:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:49:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:49:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 32, 294685), True, datetime.datetime(2023, 8, 8, 5, 49, 32, 294685), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:49:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:49:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:49:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:49:32,846 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:49:32,849 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:49:32,852 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:49:32,855 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:49:32,859 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:49:32,862 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:49:32,866 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:49:32,868 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:49:32,871 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 05:49:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:49:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:45.371682+00:00 (in 9.999465 seconds) notificationworker stdout | 2023-08-08 05:49:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:45 UTC)" (scheduled at 2023-08-08 05:49:35.371682+00:00) notificationworker stdout | 2023-08-08 05:49:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:49:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 35, 372709), True, datetime.datetime(2023, 8, 8, 5, 49, 35, 372709), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:49:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:49:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:49:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:49:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:49:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:39.579576+00:00 (in 2.999213 seconds) namespacegcworker stdout | 2023-08-08 05:49:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:36 UTC)" (scheduled at 2023-08-08 05:49:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:49:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:49:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:49:37,178 [200] [DEBUG] [app] Starting request: urn:request:394cf2a3-9f9a-4b1b-8810-5b5d17da19e1 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:49:37,179 [199] [DEBUG] [app] Starting request: urn:request:1d07f714-2f5b-4cdc-bfe6-c658ffd74f57 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:49:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:37,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:49:37,184 [203] [DEBUG] [app] Starting request: urn:request:10353ae4-1082-4409-8803-d5b2f30552ae (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:49:37,185 [210] [DEBUG] [app] Starting request: urn:request:e655d403-0de0-4b9a-81ab-3cf630b3b3bb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:49:37,185 [203] [DEBUG] [app] Ending request: urn:request:10353ae4-1082-4409-8803-d5b2f30552ae (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:10353ae4-1082-4409-8803-d5b2f30552ae', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:49:37,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:49:37,185 [210] [DEBUG] [app] Ending request: urn:request:e655d403-0de0-4b9a-81ab-3cf630b3b3bb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e655d403-0de0-4b9a-81ab-3cf630b3b3bb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:49:37,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:49:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:37,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:37,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:37,190 [197] [DEBUG] [app] Starting request: urn:request:fe04e76b-a331-413c-a832-f12c51294d42 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:37,190 [200] [DEBUG] [app] Starting request: urn:request:06016ef4-7f4f-4e5f-a567-f244b307c4b9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:37,191 [197] [DEBUG] [app] Ending request: urn:request:fe04e76b-a331-413c-a832-f12c51294d42 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fe04e76b-a331-413c-a832-f12c51294d42', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:49:37,191 [200] [DEBUG] [app] Ending request: urn:request:06016ef4-7f4f-4e5f-a567-f244b307c4b9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:06016ef4-7f4f-4e5f-a567-f244b307c4b9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:49:37,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:37,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:37,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:49:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:49:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 05:49:37,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:49:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:49:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:07.197412+00:00 (in 29.999542 seconds) buildlogsarchiver stdout | 2023-08-08 05:49:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:07 UTC)" (scheduled at 2023-08-08 05:49:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:49:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 49, 37, 198287), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:49:37,204 [200] [DEBUG] [app] Ending request: urn:request:394cf2a3-9f9a-4b1b-8810-5b5d17da19e1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:394cf2a3-9f9a-4b1b-8810-5b5d17da19e1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:49:37,204 [199] [DEBUG] [app] Ending request: urn:request:1d07f714-2f5b-4cdc-bfe6-c658ffd74f57 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1d07f714-2f5b-4cdc-bfe6-c658ffd74f57', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:49:37,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:49:37,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:49:37,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:49:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:49:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) gunicorn-web stdout | 2023-08-08 05:49:37,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:49:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:49:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 05:49:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:49:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:49:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:07 UTC)" executed successfully gcworker stdout | 2023-08-08 05:49:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:49:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:49.417660+00:00 (in 9.999472 seconds) gcworker stdout | 2023-08-08 05:49:39,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:49 UTC)" (scheduled at 2023-08-08 05:49:39.417660+00:00) gcworker stdout | 2023-08-08 05:49:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264179419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:49:39,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:49:39,430 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:49:39,430 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:49:39,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:49:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:54.393627+00:00 (in 14.813287 seconds) namespacegcworker stdout | 2023-08-08 05:49:39,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:54:39 UTC)" (scheduled at 2023-08-08 05:49:39.579576+00:00) namespacegcworker stdout | 2023-08-08 05:49:39,582 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 49, 39, 581396), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:49:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 49, 39, 581396), True, datetime.datetime(2023, 8, 8, 5, 49, 39, 581396), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:49:39,591 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 49, 39, 581396), True, datetime.datetime(2023, 8, 8, 5, 49, 39, 581396), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 5, 49, 39, 581396), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:49:39,593 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:49:39,593 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:54:39 UTC)" executed successfully builder stdout | 2023-08-08 05:49:39,756 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:49:39,765 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:49:39,765 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:49:39,770 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:49:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:49:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:46.874674+00:00 (in 4.580410 seconds) chunkcleanupworker stdout | 2023-08-08 05:49:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:52 UTC)" (scheduled at 2023-08-08 05:49:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:49:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:49:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 42, 294763), True, datetime.datetime(2023, 8, 8, 5, 49, 42, 294763), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:49:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:49:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:49:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:49:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:49:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:55.371682+00:00 (in 9.999495 seconds) notificationworker stdout | 2023-08-08 05:49:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:55 UTC)" (scheduled at 2023-08-08 05:49:45.371682+00:00) notificationworker stdout | 2023-08-08 05:49:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:49:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 45, 372685), True, datetime.datetime(2023, 8, 8, 5, 49, 45, 372685), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:49:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:49:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:49:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:49:46,874 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} chunkcleanupworker stdout | 2023-08-08 05:49:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:49:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:52.293757+00:00 (in 5.418462 seconds) chunkcleanupworker stdout | 2023-08-08 05:49:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:54:46 UTC)" (scheduled at 2023-08-08 05:49:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 05:49:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 49, 46, 875768), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:49:46,883 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 49, 46, 875768), True, datetime.datetime(2023, 8, 8, 5, 49, 46, 875768), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:49:46,885 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 49, 46, 875768), True, datetime.datetime(2023, 8, 8, 5, 49, 46, 875768), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 5, 49, 46, 875768), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:49:46,886 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:49:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:54:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:49:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:49:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:25.514341+00:00 (in 38.115243 seconds) exportactionlogsworker stdout | 2023-08-08 05:49:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:47 UTC)" (scheduled at 2023-08-08 05:49:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:49:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:49:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 47, 399597), True, datetime.datetime(2023, 8, 8, 5, 49, 47, 399597), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:49:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:49:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:49:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:49:48,073 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:49:49,272 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:49:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:49:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:59.417660+00:00 (in 9.999459 seconds) gcworker stdout | 2023-08-08 05:49:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:59 UTC)" (scheduled at 2023-08-08 05:49:49.417660+00:00) gcworker stdout | 2023-08-08 05:49:49,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264189418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:49:49,430 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:49:49,430 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:49:49,431 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:49:59 UTC)" executed successfully securityworker stdout | 2023-08-08 05:49:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:49:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:49:56.890741+00:00 (in 7.202873 seconds) securityworker stdout | 2023-08-08 05:49:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:19 UTC)" (scheduled at 2023-08-08 05:49:49.687363+00:00) securityworker stdout | 2023-08-08 05:49:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:49:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:49:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:49:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) storagereplication stdout | 2023-08-08 05:49:49,696 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityworker stdout | 2023-08-08 05:49:49,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:49:49,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:49:49,700 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:19 UTC)" executed successfully builder stdout | 2023-08-08 05:49:49,780 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:49:49,789 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:49:49,789 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:49:49,794 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:49:52,179 [197] [DEBUG] [app] Starting request: urn:request:bd5415cd-6b29-44d7-a225-69a9619efe63 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:49:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:52,180 [200] [DEBUG] [app] Starting request: urn:request:25b2ca34-26e5-46eb-bce7-88358c4a4785 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:49:52,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:52,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:49:52,185 [209] [DEBUG] [app] Starting request: urn:request:63ee967c-bc66-4739-99b4-5a30a68f6711 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:49:52,185 [209] [DEBUG] [app] Ending request: urn:request:63ee967c-bc66-4739-99b4-5a30a68f6711 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:63ee967c-bc66-4739-99b4-5a30a68f6711', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:49:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:49:52,186 [206] [DEBUG] [app] Starting request: urn:request:c678a388-727d-4cb4-b453-1cfc924af9cb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:49:52,186 [206] [DEBUG] [app] Ending request: urn:request:c678a388-727d-4cb4-b453-1cfc924af9cb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c678a388-727d-4cb4-b453-1cfc924af9cb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:49:52,187 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:49:52,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:52,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:49:52,192 [199] [DEBUG] [app] Starting request: urn:request:c4c6c87a-ec35-46cb-a56d-3e9eb9dbfe0e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:52,193 [198] [DEBUG] [app] Starting request: urn:request:d5794ace-65af-483f-8742-84a9daef4180 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:49:52,193 [199] [DEBUG] [app] Ending request: urn:request:c4c6c87a-ec35-46cb-a56d-3e9eb9dbfe0e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c4c6c87a-ec35-46cb-a56d-3e9eb9dbfe0e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:49:52,194 [198] [DEBUG] [app] Ending request: urn:request:d5794ace-65af-483f-8742-84a9daef4180 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d5794ace-65af-483f-8742-84a9daef4180', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2023-08-08 05:49:52,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:52,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:52,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:49:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:49:52,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:49:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.003) gunicorn-web stdout | 2023-08-08 05:49:52,196 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:49:52,196 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:49:52,200 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:49:52,201 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:49:52,206 [200] [DEBUG] [app] Ending request: urn:request:25b2ca34-26e5-46eb-bce7-88358c4a4785 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:25b2ca34-26e5-46eb-bce7-88358c4a4785', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:49:52,206 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:49:52,207 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:49:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:49:52,207 [197] [DEBUG] [app] Ending request: urn:request:bd5415cd-6b29-44d7-a225-69a9619efe63 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bd5415cd-6b29-44d7-a225-69a9619efe63', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:49:52,207 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:49:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) gunicorn-web stdout | 2023-08-08 05:49:52,207 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:49:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:49:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.029) chunkcleanupworker stdout | 2023-08-08 05:49:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:49:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:02.293757+00:00 (in 9.999428 seconds) chunkcleanupworker stdout | 2023-08-08 05:49:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:02 UTC)" (scheduled at 2023-08-08 05:49:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:49:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:49:52,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 52, 295000), True, datetime.datetime(2023, 8, 8, 5, 49, 52, 295000), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:49:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:49:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:49:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:49:52,503 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:49:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:49:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:13.097375+00:00 (in 18.812742 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:49:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:54 UTC)" (scheduled at 2023-08-08 05:49:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:49:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:49:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:49:54,303 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:49:54,385 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} namespacegcworker stdout | 2023-08-08 05:49:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:49:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:36.579841+00:00 (in 42.185163 seconds) namespacegcworker stdout | 2023-08-08 05:49:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:54 UTC)" (scheduled at 2023-08-08 05:49:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:49:54,395 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:49:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 54, 395126), True, datetime.datetime(2023, 8, 8, 5, 49, 54, 395126), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:49:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:49:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:49:54,404 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:50:54 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:49:54,620 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:49:54,666 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:49:54,669 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:49:54,673 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:49:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:49:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:02.178530+00:00 (in 6.806328 seconds) notificationworker stdout | 2023-08-08 05:49:55,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:05 UTC)" (scheduled at 2023-08-08 05:49:55.371682+00:00) notificationworker stdout | 2023-08-08 05:49:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:49:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 49, 55, 373116), True, datetime.datetime(2023, 8, 8, 5, 49, 55, 373116), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:49:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:49:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:49:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:49:55,759 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:49:56,287 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:49:56,496 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:49:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:49:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:19.687363+00:00 (in 22.796094 seconds) securityworker stdout | 2023-08-08 05:49:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:26 UTC)" (scheduled at 2023-08-08 05:49:56.890741+00:00) securityworker stdout | 2023-08-08 05:49:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:49:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:49:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:49:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:49:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:49:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:49:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:49:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:49:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:49:56,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:49:56,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:49:56,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:49:56,904 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:49:56,904 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:49:56,904 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:49:56,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:49:56,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:49:56,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:49:57,052 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:49:57,054 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:49:57,057 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:49:57,060 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:49:57,063 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:49:57,721 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:49:57,793 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:49:58,932 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:49:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:49:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:09.417660+00:00 (in 9.999450 seconds) gcworker stdout | 2023-08-08 05:49:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:09 UTC)" (scheduled at 2023-08-08 05:49:59.417660+00:00) gcworker stdout | 2023-08-08 05:49:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264199418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:49:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:49:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:49:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:49:59,762 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} builder stdout | 2023-08-08 05:49:59,804 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue servicekey stdout | 2023-08-08 05:49:59,804 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:49:59,814 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:49:59,814 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:49:59,819 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:50:00,300 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:50:00,905 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:50:01,150 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:50:01,433 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:50:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:50:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:05.371682+00:00 (in 3.192642 seconds) notificationworker stdout | 2023-08-08 05:50:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:02 UTC)" (scheduled at 2023-08-08 05:50:02.178530+00:00) notificationworker stdout | 2023-08-08 05:50:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:50:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:50:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:50:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:12.293757+00:00 (in 9.999456 seconds) chunkcleanupworker stdout | 2023-08-08 05:50:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:12 UTC)" (scheduled at 2023-08-08 05:50:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:50:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:50:02,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 2, 294830), True, datetime.datetime(2023, 8, 8, 5, 50, 2, 294830), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:50:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:50:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:50:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:50:02,857 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:50:02,862 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:50:02,866 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:50:02,872 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:50:02,877 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:50:02,880 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:50:02,883 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:50:02,886 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:50:02,890 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:50:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:50:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:09.793658+00:00 (in 5.605725 seconds) repositorygcworker stdout | 2023-08-08 05:50:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:04 UTC)" (scheduled at 2023-08-08 05:50:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:50:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:50:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:50:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:50:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:15.371682+00:00 (in 9.999495 seconds) notificationworker stdout | 2023-08-08 05:50:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:15 UTC)" (scheduled at 2023-08-08 05:50:05.371682+00:00) notificationworker stdout | 2023-08-08 05:50:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:50:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 5, 372897), True, datetime.datetime(2023, 8, 8, 5, 50, 5, 372897), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:50:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:50:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:50:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:05,778 [198] [DEBUG] [app] Starting request: urn:request:339af69b-40cb-4115-953e-0607f677e469 (/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:05,788 [198] [DEBUG] [endpoints.api.discovery] Unable to find method for POST in class gunicorn-web stdout | 2023-08-08 05:50:05,793 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:05,813 [198] [DEBUG] [app] Ending request: urn:request:339af69b-40cb-4115-953e-0607f677e469 (/) {'endpoint': 'web.index', 'request_id': 'urn:request:339af69b-40cb-4115-953e-0607f677e469', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/', 'path': '/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:05,814 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:05 +0000] "GET / HTTP/1.0" 200 85192 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:05 +0000] "GET / HTTP/2.0" 200 13315 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.038 726 0.037) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:06 +0000] "GET /static/img/network-tile.png HTTP/2.0" 200 6289 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 507 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:06 +0000] "GET /static/build/489e5e00e1a9a80f88d55281894a87ff.woff HTTP/2.0" 200 7280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 139 -) gunicorn-web stdout | 2023-08-08 05:50:06,792 [199] [DEBUG] [app] Starting request: urn:request:26b5e674-2073-47f2-adbf-a9ca8255129c (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:06,792 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,792 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,793 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:06,800 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:06,800 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:06,800 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,800 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,800 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:06,800 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:06,800 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:06,801 [199] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 05:50:06,804 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,805 [199] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 05:50:06,807 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:50:06,808 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:06,809 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:50:06,811 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:06,813 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691473806812, 0, 1, None]) gunicorn-web stdout | 2023-08-08 05:50:06,816 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:06,818 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:50:06,819 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:06,821 [199] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691473806820, 0, 1]) gunicorn-web stdout | 2023-08-08 05:50:06,822 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,822 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,822 [200] [DEBUG] [app] Starting request: urn:request:2076ad8d-aeb8-4ed1-9f5d-b70e77155bac (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:06,822 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,822 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,823 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:06,823 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:50:06,825 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:06,825 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:06,825 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:50:06,825 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:06,825 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,825 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,825 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,826 [199] [DEBUG] [app] Ending request: urn:request:26b5e674-2073-47f2-adbf-a9ca8255129c (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:26b5e674-2073-47f2-adbf-a9ca8255129c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:06,827 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:06 +0000] "GET /api/v1/user/ HTTP/2.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.036 166 0.035) gunicorn-web stdout | 2023-08-08 05:50:06,827 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:06 +0000] "GET /api/v1/user/ HTTP/1.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:06,830 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:06,830 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:06,830 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:06,831 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 05:50:06,833 [200] [DEBUG] [app] Ending request: urn:request:2076ad8d-aeb8-4ed1-9f5d-b70e77155bac (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:2076ad8d-aeb8-4ed1-9f5d-b70e77155bac', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:06,834 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:06 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.015 47 0.014) gunicorn-web stdout | 2023-08-08 05:50:06,835 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:06 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:07,179 [200] [DEBUG] [app] Starting request: urn:request:a1bd98e5-37d8-4780-ab92-091decd28413 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:50:07,180 [197] [DEBUG] [app] Starting request: urn:request:77f5328e-36bb-49b4-a5d3-9d65ce4c56b0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:50:07,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:50:07,184 [203] [DEBUG] [app] Starting request: urn:request:379467f8-7828-419d-81bc-8f40e72773bd (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:50:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:50:07,185 [203] [DEBUG] [app] Ending request: urn:request:379467f8-7828-419d-81bc-8f40e72773bd (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:379467f8-7828-419d-81bc-8f40e72773bd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:50:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:50:07,185 [207] [DEBUG] [app] Starting request: urn:request:bb77b8c3-b98d-49bc-bdea-d3182a1d1c5a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:50:07,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:50:07,186 [207] [DEBUG] [app] Ending request: urn:request:bb77b8c3-b98d-49bc-bdea-d3182a1d1c5a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:bb77b8c3-b98d-49bc-bdea-d3182a1d1c5a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:50:07,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:07,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:07,190 [197] [DEBUG] [app] Starting request: urn:request:edbd3e99-0217-4fbf-85fa-1b8ba50ce9d9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:50:07,191 [197] [DEBUG] [app] Ending request: urn:request:edbd3e99-0217-4fbf-85fa-1b8ba50ce9d9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:edbd3e99-0217-4fbf-85fa-1b8ba50ce9d9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:50:07,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:07,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:07,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:50:07,193 [199] [DEBUG] [app] Starting request: urn:request:6b204de2-8276-48d2-91cf-ebbf87e803c0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:50:07,193 [199] [DEBUG] [app] Ending request: urn:request:6b204de2-8276-48d2-91cf-ebbf87e803c0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6b204de2-8276-48d2-91cf-ebbf87e803c0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:50:07,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:07,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:07,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:50:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:50:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:50:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:37.197412+00:00 (in 29.999523 seconds) buildlogsarchiver stdout | 2023-08-08 05:50:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:37 UTC)" (scheduled at 2023-08-08 05:50:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:50:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 50, 7, 198291), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:50:07,203 [200] [DEBUG] [app] Ending request: urn:request:a1bd98e5-37d8-4780-ab92-091decd28413 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a1bd98e5-37d8-4780-ab92-091decd28413', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:50:07,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 05:50:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:50:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:50:07,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:50:07,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:50:07,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:37 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:07,205 [197] [DEBUG] [app] Ending request: urn:request:77f5328e-36bb-49b4-a5d3-9d65ce4c56b0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:77f5328e-36bb-49b4-a5d3-9d65ce4c56b0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:50:07,206 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:07,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:50:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) gunicorn-web stdout | 2023-08-08 05:50:07,271 [200] [DEBUG] [app] Starting request: urn:request:6b7b4093-ce4e-4677-997f-844a93a2b459 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:07,271 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,271 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,272 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,272 [199] [DEBUG] [app] Starting request: urn:request:73f58a29-4df3-40aa-9a0a-2de018e6a72f (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:07,273 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,273 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,273 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,279 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:07,280 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:07,280 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,280 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:50:07,280 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,280 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,280 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,280 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,281 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:07,281 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:07,281 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,281 [199] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:50:07,281 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,281 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,281 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,281 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,282 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 2, 7, 282027), datetime.datetime(2023, 8, 8, 5, 50, 7, 282098)]) gunicorn-web stdout | 2023-08-08 05:50:07,283 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 05:50:07,284 [199] [DEBUG] [app] Ending request: urn:request:73f58a29-4df3-40aa-9a0a-2de018e6a72f (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:73f58a29-4df3-40aa-9a0a-2de018e6a72f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:07,285 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.014 62 0.013) gunicorn-web stdout | 2023-08-08 05:50:07,285 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:07,287 [200] [DEBUG] [app] Ending request: urn:request:6b7b4093-ce4e-4677-997f-844a93a2b459 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:6b7b4093-ce4e-4677-997f-844a93a2b459', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:07,288 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.019 398 0.019) gunicorn-web stdout | 2023-08-08 05:50:07,289 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:07,325 [200] [DEBUG] [app] Starting request: urn:request:1c2bb7f2-8c39-4a84-b1d3-8d069e447901 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:07,325 [199] [DEBUG] [app] Starting request: urn:request:83b2205a-0180-430d-b9c1-fd9fed585e97 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:07,325 [198] [DEBUG] [app] Starting request: urn:request:8622d5d6-fcda-4189-a2c0-3a1e1d13213b (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:07,325 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,325 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,325 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,325 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,326 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,326 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,326 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,326 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,326 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,332 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:07,332 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:07,333 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,333 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:07,333 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:07,333 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,333 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,333 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,333 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,333 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,333 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:07,333 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:07,333 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,333 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:07,333 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,333 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:07,333 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,333 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:07,333 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:07,334 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,334 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,335 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:50:07,335 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:50:07,336 [198] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'build_test', 101]) gunicorn-web stdout | 2023-08-08 05:50:07,337 [199] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'whuquay', 101]) gunicorn-web stdout | 2023-08-08 05:50:07,338 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:50:07,339 [198] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Max("t1"."lifetime_start_ms") FROM "tag" AS "t1" WHERE ((("t1"."repository_id" IN (%s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) GROUP BY "t1"."repository_id"', [1, None, 1691473807338, False]) gunicorn-web stdout | 2023-08-08 05:50:07,340 [200] [DEBUG] [app] Ending request: urn:request:1c2bb7f2-8c39-4a84-b1d3-8d069e447901 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:1c2bb7f2-8c39-4a84-b1d3-8d069e447901', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:07,340 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:50:07,340 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:07,341 [198] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Sum("t1"."count") FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" IN (%s)) AND ("t1"."date" >= %s)) GROUP BY "t1"."repository_id"', [1, datetime.date(2023, 8, 1)]) gunicorn-web stdout | 2023-08-08 05:50:07,341 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 156 0.017) gunicorn-web stdout | 2023-08-08 05:50:07,343 [199] [DEBUG] [app] Ending request: urn:request:83b2205a-0180-430d-b9c1-fd9fed585e97 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:83b2205a-0180-430d-b9c1-fd9fed585e97', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'whuquay', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:07,343 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:50:07,343 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 103 0.020) gunicorn-web stdout | 2023-08-08 05:50:07,344 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:07,345 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,347 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 05:50:07,348 [198] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,350 [198] [DEBUG] [peewee] ('SELECT Sum("t1"."layers_compressed_size") AS "size_bytes" FROM "manifest" AS "t1" INNER JOIN (SELECT "t2"."manifest_id" FROM "tag" AS "t2" WHERE ((("t2"."hidden" = %s) AND (("t2"."lifetime_end_ms" IS %s) OR ("t2"."lifetime_end_ms" > %s))) AND ("t2"."repository_id" = %s)) GROUP BY "t2"."manifest_id" HAVING (Count("t2"."name") > %s)) AS "t3" ON ("t3"."manifest_id" = "t1"."id") WHERE ("t1"."repository_id" = %s)', [False, None, 1691473807349, 1, 0, 1]) gunicorn-web stdout | 2023-08-08 05:50:07,351 [198] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,352 [198] [DEBUG] [peewee] ('UPDATE "repositorysize" SET "size_bytes" = %s WHERE ("repositorysize"."repository_id" = %s)', [0, 1]) gunicorn-web stdout | 2023-08-08 05:50:07,360 [198] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:07,361 [198] [DEBUG] [app] Ending request: urn:request:8622d5d6-fcda-4189-a2c0-3a1e1d13213b (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:8622d5d6-fcda-4189-a2c0-3a1e1d13213b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'false', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:07,362 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:07,362 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/1.0" 200 274 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:07 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/2.0" 200 274 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.038 114 0.039) gcworker stdout | 2023-08-08 05:50:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:50:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:19.417660+00:00 (in 9.999419 seconds) gcworker stdout | 2023-08-08 05:50:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:19 UTC)" (scheduled at 2023-08-08 05:50:09.417660+00:00) gcworker stdout | 2023-08-08 05:50:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264209418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:50:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:50:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:50:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:50:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:50:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:04.187429+00:00 (in 54.393215 seconds) repositorygcworker stdout | 2023-08-08 05:50:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:09 UTC)" (scheduled at 2023-08-08 05:50:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:50:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:50:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 9, 794664), True, datetime.datetime(2023, 8, 8, 5, 50, 9, 794664), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:50:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:50:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:50:09,804 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:09 UTC)" executed successfully builder stdout | 2023-08-08 05:50:09,819 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:50:09,828 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:50:09,828 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:50:09,832 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:12,111 [197] [DEBUG] [app] Starting request: urn:request:d71355d1-f7e7-4164-bd74-90edf6284fb2 (/api/v1/repository/build_test/gitlab) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:12,111 [199] [DEBUG] [app] Starting request: urn:request:c2b0847b-a59a-4c48-933d-e38cacb68d53 (/api/v1/repository/build_test/gitlab/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:12,111 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,111 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,111 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,111 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,112 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,112 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,119 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:12,119 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:12,119 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:12,119 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:12,119 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,119 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,119 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:50:12,119 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:50:12,119 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,119 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,119 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:12,119 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:12,119 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:12,119 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:12,119 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:12,119 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:12,122 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:12,122 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:12,128 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:50:12,128 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:50:12,128 [197] [DEBUG] [endpoints.api.repository] Get repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:50:12,129 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,129 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,131 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [1, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,132 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [1, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:50:12,132 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,133 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,134 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:12,134 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,134 [197] [DEBUG] [app] Ending request: urn:request:d71355d1-f7e7-4164-bd74-90edf6284fb2 (/api/v1/repository/build_test/gitlab) {'endpoint': 'api.repository', 'request_id': 'urn:request:d71355d1-f7e7-4164-bd74-90edf6284fb2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/gitlab', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:12,135 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:12,136 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [1, None, 1691473812135, False, 101, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,136 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:12 +0000] "GET /api/v1/repository/build_test/gitlab?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:12 +0000] "GET /api/v1/repository/build_test/gitlab?includeStats=true&includeTags=false HTTP/2.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 87 0.027) gunicorn-web stdout | 2023-08-08 05:50:12,138 [199] [DEBUG] [app] Ending request: urn:request:c2b0847b-a59a-4c48-933d-e38cacb68d53 (/api/v1/repository/build_test/gitlab/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:c2b0847b-a59a-4c48-933d-e38cacb68d53', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/gitlab/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:12,139 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:12 +0000] "GET /api/v1/repository/build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 521 0.030) gunicorn-web stdout | 2023-08-08 05:50:12,139 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:12 +0000] "GET /api/v1/repository/build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" chunkcleanupworker stdout | 2023-08-08 05:50:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:50:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:22.293757+00:00 (in 9.999497 seconds) chunkcleanupworker stdout | 2023-08-08 05:50:12,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:22 UTC)" (scheduled at 2023-08-08 05:50:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:50:12,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:50:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 12, 295228), True, datetime.datetime(2023, 8, 8, 5, 50, 12, 295228), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:50:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:50:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:50:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:22 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:12,958 [198] [DEBUG] [app] Starting request: urn:request:30ca5ed5-8725-432a-b8a3-6a1dfd08b688 (/repository/build_test/gitlab/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:12,960 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,967 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,968 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,972 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['gitlab', 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:12,978 [198] [DEBUG] [app] Ending request: urn:request:30ca5ed5-8725-432a-b8a3-6a1dfd08b688 (/repository/build_test/gitlab/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:30ca5ed5-8725-432a-b8a3-6a1dfd08b688', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab/status?token=0fe857a5-447d-43a8-9f00-b707338bb0d3', 'path': '/repository/build_test/gitlab/status', 'parameters': {'token': '0fe857a5-447d-43a8-9f00-b707338bb0d3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:12,979 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:12,979 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:12 +0000] "GET /repository/build_test/gitlab/status?token=0fe857a5-447d-43a8-9f00-b707338bb0d3 HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:12 +0000] "GET /repository/build_test/gitlab/status?token=0fe857a5-447d-43a8-9f00-b707338bb0d3 HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.022 436 0.021) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:13 +0000] "GET /static/fonts/glyphicons-halflings-regular.woff2 HTTP/2.0" 200 18028 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/static/ldn/bootstrap.min.css-13f668494557.css" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 292 -) securityscanningnotificationworker stdout | 2023-08-08 05:50:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:50:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:54.284093+00:00 (in 41.186059 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:50:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:13 UTC)" (scheduled at 2023-08-08 05:50:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:50:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:50:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 13, 98655), True, datetime.datetime(2023, 8, 8, 5, 50, 13, 98655), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:50:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:50:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:50:13,108 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:13 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:13,129 [198] [DEBUG] [app] Starting request: urn:request:ffeda7df-7242-4d1a-9f16-3b5dd46a3552 (/api/v1/repository/build_test/gitlab/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:13,129 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:13,129 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:13,130 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:13,137 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:13,137 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:13,137 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:13,137 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:50:13,137 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:13,137 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:13,137 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:13,137 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:13,139 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:13,146 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:50:13,147 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:13,151 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['gitlab', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:50:13,155 [198] [DEBUG] [app] Ending request: urn:request:ffeda7df-7242-4d1a-9f16-3b5dd46a3552 (/api/v1/repository/build_test/gitlab/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:ffeda7df-7242-4d1a-9f16-3b5dd46a3552', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/build/?limit=3', 'path': '/api/v1/repository/build_test/gitlab/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:13,155 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:13,156 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:13 +0000] "GET /api/v1/repository/build_test/gitlab/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:13 +0000] "GET /api/v1/repository/build_test/gitlab/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 88 0.028) notificationworker stdout | 2023-08-08 05:50:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:50:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:25.371682+00:00 (in 9.999438 seconds) notificationworker stdout | 2023-08-08 05:50:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:25 UTC)" (scheduled at 2023-08-08 05:50:15.371682+00:00) notificationworker stdout | 2023-08-08 05:50:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:50:15,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 15, 372987), True, datetime.datetime(2023, 8, 8, 5, 50, 15, 372987), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:50:15,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:50:15,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:50:15,383 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:16,233 [198] [DEBUG] [app] Starting request: urn:request:d061e26e-f8c9-4bb5-ab72-8cc67d10fc1b (/api/v1/repository/build_test/gitlab/permissions/team/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:16,233 [197] [DEBUG] [app] Starting request: urn:request:5904c05d-a18d-4dde-b553-ff6bc62ee382 (/api/v1/repository/build_test/gitlab/permissions/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:16,233 [199] [DEBUG] [app] Starting request: urn:request:311a51f9-af7a-427a-9a66-01af771ef0ae (/api/v1/repository/build_test/gitlab/notification/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:16,233 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,233 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,233 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,233 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,233 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,233 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,234 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:16,234 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:16,234 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:16,241 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:16,241 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:16,241 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,241 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:16,241 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:16,241 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:50:16,241 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,241 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,241 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,241 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,241 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,241 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:50:16,241 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,242 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,242 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,242 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:16,242 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,242 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:16,242 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,242 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:50:16,242 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:16,242 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,242 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,242 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,244 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:16,244 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:16,244 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:16,250 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,250 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,251 [198] [DEBUG] [peewee] ('SELECT "t1"."name", "t2"."name", "t3"."id", "t3"."team_id", "t3"."user_id", "t3"."repository_id", "t3"."role_id" FROM "repositorypermission" AS "t3" INNER JOIN "team" AS "t1" ON ("t3"."team_id" = "t1"."id") INNER JOIN "role" AS "t2" ON ("t3"."role_id" = "t2"."id") INNER JOIN "repository" AS "t4" ON ("t3"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") WHERE (("t5"."username" = %s) AND ("t4"."name" = %s))', ['build_test', 'gitlab']) gunicorn-web stdout | 2023-08-08 05:50:16,251 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:16,251 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:50:16,252 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:16,253 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."name", "t3"."id", "t3"."team_id", "t3"."user_id", "t3"."repository_id", "t3"."role_id" FROM "repositorypermission" AS "t3" INNER JOIN "user" AS "t1" ON ("t3"."user_id" = "t1"."id") INNER JOIN "role" AS "t2" ON ("t3"."role_id" = "t2"."id") INNER JOIN "repository" AS "t4" ON ("t3"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") WHERE (("t5"."username" = %s) AND ("t4"."name" = %s))', ['build_test', 'gitlab']) gunicorn-web stdout | 2023-08-08 05:50:16,253 [198] [DEBUG] [app] Ending request: urn:request:d061e26e-f8c9-4bb5-ab72-8cc67d10fc1b (/api/v1/repository/build_test/gitlab/permissions/team/) {'endpoint': 'api.repositoryteampermissionlist', 'request_id': 'urn:request:d061e26e-f8c9-4bb5-ab72-8cc67d10fc1b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/permissions/team/', 'path': '/api/v1/repository/build_test/gitlab/permissions/team/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:16,254 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:16,254 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:16 +0000] "GET /api/v1/repository/build_test/gitlab/permissions/team/ HTTP/1.0" 200 20 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:16 +0000] "GET /api/v1/repository/build_test/gitlab/permissions/team/ HTTP/2.0" 200 20 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.023 74 0.022) gunicorn-web stdout | 2023-08-08 05:50:16,255 [197] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."username" FROM "user" AS "t1" INNER JOIN "teammember" AS "t2" ON ("t2"."user_id" = "t1"."id") INNER JOIN "team" AS "t3" ON ("t2"."team_id" = "t3"."id") WHERE ((("t3"."organization_id" = %s) AND ("t1"."robot" = %s)) AND ("t1"."id" IN (%s)))', [2, False, 1]) gunicorn-web stdout | 2023-08-08 05:50:16,255 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."event_id", "t1"."method_id", "t1"."title", "t1"."config_json", "t1"."event_config_json", "t1"."number_of_failures", "t2"."id", "t2"."namespace_user_id", "t2"."name", "t2"."visibility_id", "t2"."description", "t2"."badge_token", "t2"."kind_id", "t2"."trust_enabled", "t2"."state", "t3"."id", "t3"."uuid", "t3"."username", "t3"."password_hash", "t3"."email", "t3"."verified", "t3"."stripe_id", "t3"."organization", "t3"."robot", "t3"."invoice_email", "t3"."invalid_login_attempts", "t3"."last_invalid_login", "t3"."removed_tag_expiration_s", "t3"."enabled", "t3"."invoice_email_address", "t3"."given_name", "t3"."family_name", "t3"."company", "t3"."location", "t3"."maximum_queued_builds_count", "t3"."creation_date", "t3"."last_accessed" FROM "repositorynotification" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE (("t3"."username" = %s) AND ("t2"."name" = %s))', ['build_test', 'gitlab']) gunicorn-web stdout | 2023-08-08 05:50:16,257 [197] [DEBUG] [app] Ending request: urn:request:5904c05d-a18d-4dde-b553-ff6bc62ee382 (/api/v1/repository/build_test/gitlab/permissions/user/) {'endpoint': 'api.repositoryuserpermissionlist', 'request_id': 'urn:request:5904c05d-a18d-4dde-b553-ff6bc62ee382', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/permissions/user/', 'path': '/api/v1/repository/build_test/gitlab/permissions/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:16,257 [199] [DEBUG] [app] Ending request: urn:request:311a51f9-af7a-427a-9a66-01af771ef0ae (/api/v1/repository/build_test/gitlab/notification/) {'endpoint': 'api.repositorynotificationlist', 'request_id': 'urn:request:311a51f9-af7a-427a-9a66-01af771ef0ae', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab/notification/', 'path': '/api/v1/repository/build_test/gitlab/notification/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:16,258 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:16,258 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:16,258 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:16 +0000] "GET /api/v1/repository/build_test/gitlab/permissions/user/ HTTP/1.0" 200 252 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:16 +0000] "GET /api/v1/repository/build_test/gitlab/permissions/user/ HTTP/2.0" 200 252 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 567 0.026) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:16 +0000] "GET /api/v1/repository/build_test/gitlab/notification/ HTTP/2.0" 200 22 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 71 0.027) gunicorn-web stdout | 2023-08-08 05:50:16,258 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:16 +0000] "GET /api/v1/repository/build_test/gitlab/notification/ HTTP/1.0" 200 22 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 05:50:16,912 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:50:18,110 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:50:19,301 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:50:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:50:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:29.417660+00:00 (in 9.999500 seconds) gcworker stdout | 2023-08-08 05:50:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:29 UTC)" (scheduled at 2023-08-08 05:50:19.417660+00:00) gcworker stdout | 2023-08-08 05:50:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264219418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:50:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:50:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:50:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:29 UTC)" executed successfully securityworker stdout | 2023-08-08 05:50:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:50:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:26.890741+00:00 (in 7.202852 seconds) securityworker stdout | 2023-08-08 05:50:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:49 UTC)" (scheduled at 2023-08-08 05:50:19.687363+00:00) securityworker stdout | 2023-08-08 05:50:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:50:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:50:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:50:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:50:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:50:19,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:50:19,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:49 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:50:19,711 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:50:19,835 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:50:19,844 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:50:19,844 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:50:19,848 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:22,179 [199] [DEBUG] [app] Starting request: urn:request:e30e6a7f-2b12-493f-b005-d1aca767859d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:50:22,179 [200] [DEBUG] [app] Starting request: urn:request:727b6b78-f7cf-4d60-b22d-be9c44120e77 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:50:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:50:22,185 [209] [DEBUG] [app] Starting request: urn:request:ccd1db6b-2225-48b1-97d1-17f6c90bdf02 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:50:22,185 [210] [DEBUG] [app] Starting request: urn:request:ae7ccb6c-324c-4408-b692-aa83e7643e54 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:50:22,186 [209] [DEBUG] [app] Ending request: urn:request:ccd1db6b-2225-48b1-97d1-17f6c90bdf02 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ccd1db6b-2225-48b1-97d1-17f6c90bdf02', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:50:22,186 [210] [DEBUG] [app] Ending request: urn:request:ae7ccb6c-324c-4408-b692-aa83e7643e54 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ae7ccb6c-324c-4408-b692-aa83e7643e54', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:50:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:50:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:22,192 [199] [DEBUG] [app] Starting request: urn:request:e71ff3d8-4d33-44c0-bf9e-44c111a8bc89 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:50:22,192 [199] [DEBUG] [app] Ending request: urn:request:e71ff3d8-4d33-44c0-bf9e-44c111a8bc89 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e71ff3d8-4d33-44c0-bf9e-44c111a8bc89', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:50:22,193 [198] [DEBUG] [app] Starting request: urn:request:7927746b-3523-4c7d-aaf4-38c317166577 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:50:22,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:22,193 [198] [DEBUG] [app] Ending request: urn:request:7927746b-3523-4c7d-aaf4-38c317166577 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7927746b-3523-4c7d-aaf4-38c317166577', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:50:22,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2023-08-08 05:50:22,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:22,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:50:22,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:50:22,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:50:22,205 [200] [DEBUG] [app] Ending request: urn:request:727b6b78-f7cf-4d60-b22d-be9c44120e77 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:727b6b78-f7cf-4d60-b22d-be9c44120e77', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:50:22,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:22,205 [199] [DEBUG] [app] Ending request: urn:request:e30e6a7f-2b12-493f-b005-d1aca767859d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e30e6a7f-2b12-493f-b005-d1aca767859d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:50:22,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:22,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:50:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:50:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:50:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) gunicorn-web stdout | 2023-08-08 05:50:22,206 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:50:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:50:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:50:22,295 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:28.874906+00:00 (in 6.580143 seconds) chunkcleanupworker stdout | 2023-08-08 05:50:22,296 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:32 UTC)" (scheduled at 2023-08-08 05:50:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:50:22,296 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:50:22,297 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 22, 296424), True, datetime.datetime(2023, 8, 8, 5, 50, 22, 296424), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:50:22,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:50:22,305 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:50:22,305 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:50:22,524 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:50:24,317 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:50:24,420 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:50:24,657 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-secscan stdout | 2023-08-08 05:50:24,673 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:50:24,676 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:50:24,680 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 05:50:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:50:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:35.371682+00:00 (in 9.999428 seconds) notificationworker stdout | 2023-08-08 05:50:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:35 UTC)" (scheduled at 2023-08-08 05:50:25.371682+00:00) notificationworker stdout | 2023-08-08 05:50:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:50:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 25, 372869), True, datetime.datetime(2023, 8, 8, 5, 50, 25, 372869), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:50:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:50:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:50:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:50:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:50:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:47.398551+00:00 (in 21.883653 seconds) exportactionlogsworker stdout | 2023-08-08 05:50:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:25 UTC)" (scheduled at 2023-08-08 05:50:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:50:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:50:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:50:25,778 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:50:26,318 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:50:26,507 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:50:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:50:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:49.687363+00:00 (in 22.795959 seconds) securityworker stdout | 2023-08-08 05:50:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:56 UTC)" (scheduled at 2023-08-08 05:50:26.890741+00:00) securityworker stdout | 2023-08-08 05:50:26,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:50:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:50:26,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:50:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:50:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:50:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:50:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:50:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:50:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:50:26,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:50:26,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:50:26,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:50:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:50:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:50:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:50:26,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:50:26,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:50:26,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:50:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:27,063 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:50:27,067 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:50:27,070 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:50:27,074 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:50:27,076 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:50:27,746 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:50:27,827 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gunicorn-web stdout | 2023-08-08 05:50:28,212 [199] [DEBUG] [app] Starting request: urn:request:b8882eae-f0a9-4e6e-a723-4b2fcaafcf31 (/api/v1/repository/build_test/gitlab) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:28,213 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:28,213 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:28,214 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:28,222 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:28,222 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:28,222 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:28,222 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/gitlab gunicorn-web stdout | 2023-08-08 05:50:28,222 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:28,222 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:28,222 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:28,222 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:28,225 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'build_test', 1, 'gitlab', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:28,231 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 05:50:28,232 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:28,234 [199] [DEBUG] [peewee] ('DELETE FROM "star" WHERE ("star"."repository_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:50:28,235 [199] [DEBUG] [peewee] ('UPDATE "repository" SET "namespace_user_id" = %s, "name" = %s, "visibility_id" = %s, "description" = %s, "badge_token" = %s, "kind_id" = %s, "trust_enabled" = %s, "state" = %s WHERE ("repository"."id" = %s)', [2, 'ec080677-2434-4b6f-b264-cdeda28a0087', 2, '', '0fe857a5-447d-43a8-9f00-b707338bb0d3', 1, False, 3, 1]) gunicorn-web stdout | 2023-08-08 05:50:28,237 [199] [DEBUG] [peewee] ('INSERT INTO "deletedrepository" ("repository_id", "marked", "original_name") VALUES (%s, %s, %s) RETURNING "deletedrepository"."id"', [1, datetime.datetime(2023, 8, 8, 5, 50, 28, 236974), 'gitlab']) gunicorn-web stdout | 2023-08-08 05:50:28,239 [199] [DEBUG] [peewee] ('INSERT INTO "queueitem" ("queue_name", "body", "available_after", "available", "retries_remaining", "state_id") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "queueitem"."id"', ['repositorygc/build_test/1/', '{"marker_id": 1, "original_name": "gitlab"}', datetime.datetime(2023, 8, 8, 5, 50, 28, 238886), True, 5, 'bb4cc934-5b95-44f5-845c-0b3038d59ca6']) gunicorn-web stdout | 2023-08-08 05:50:28,240 [199] [DEBUG] [peewee] ('UPDATE "deletedrepository" SET "repository_id" = %s, "marked" = %s, "original_name" = %s, "queue_id" = %s WHERE ("deletedrepository"."id" = %s)', [1, datetime.datetime(2023, 8, 8, 5, 50, 28, 236974), 'gitlab', '1', 1]) gunicorn-web stdout | 2023-08-08 05:50:28,243 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:28,244 [199] [DEBUG] [peewee] ('DELETE FROM "queueitem" WHERE ("queueitem"."queue_name" ILIKE %s)', ['dockerfilebuild/build_test/gitlab/%']) gunicorn-web stdout | 2023-08-08 05:50:28,246 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:28,247 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "logentrykind" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:50:28,251 [199] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [19, 2, 1, None, datetime.datetime(2023, 8, 8, 5, 50, 28, 245651), '10.130.2.1', '{"repo": "gitlab", "namespace": "build_test"}']) gunicorn-web stdout | 2023-08-08 05:50:28,254 [199] [DEBUG] [app] Ending request: urn:request:b8882eae-f0a9-4e6e-a723-4b2fcaafcf31 (/api/v1/repository/build_test/gitlab) {'endpoint': 'api.repository', 'request_id': 'urn:request:b8882eae-f0a9-4e6e-a723-4b2fcaafcf31', 'remote_addr': '10.130.2.1', 'http_method': 'DELETE', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/gitlab', 'path': '/api/v1/repository/build_test/gitlab', 'parameters': {}, 'json_body': {}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:28,255 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:28 +0000] "DELETE /api/v1/repository/build_test/gitlab HTTP/2.0" 204 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.045 959 0.044) gunicorn-web stdout | 2023-08-08 05:50:28,255 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:28 +0000] "DELETE /api/v1/repository/build_test/gitlab HTTP/1.0" 204 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:28,761 [198] [DEBUG] [app] Starting request: urn:request:124163f5-a879-404b-998a-23e165f91df0 (/repository/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:28,763 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:28,765 [198] [DEBUG] [app] Ending request: urn:request:124163f5-a879-404b-998a-23e165f91df0 (/repository/) {'endpoint': 'web.repository', 'request_id': 'urn:request:124163f5-a879-404b-998a-23e165f91df0', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/', 'path': '/repository/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:28,766 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:28 +0000] "GET /repository/ HTTP/1.0" 200 85192 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:28 +0000] "GET /repository/ HTTP/2.0" 200 13315 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/gitlab?tab=settings" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.006 506 0.006) chunkcleanupworker stdout | 2023-08-08 05:50:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:50:28,876 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:32.293757+00:00 (in 3.417833 seconds) chunkcleanupworker stdout | 2023-08-08 05:50:28,876 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:28 UTC)" (scheduled at 2023-08-08 05:50:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:50:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:50:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:50:28,945 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:50:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:50:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:39.417660+00:00 (in 9.999483 seconds) gcworker stdout | 2023-08-08 05:50:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:39 UTC)" (scheduled at 2023-08-08 05:50:29.417660+00:00) gcworker stdout | 2023-08-08 05:50:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264229418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:50:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:50:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:50:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:39 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:29 +0000] "GET /static/img/quay_favicon.png HTTP/2.0" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 160 -) gunicorn-web stdout | 2023-08-08 05:50:29,651 [199] [DEBUG] [app] Starting request: urn:request:62a92b94-3297-4ca2-a0b2-a9e98a16dd12 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:29,651 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,651 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,652 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:29,658 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:29,659 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:29,659 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,659 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,659 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:29,659 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:29,659 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:29,660 [199] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 05:50:29,663 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,663 [199] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 05:50:29,665 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:50:29,666 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:29,667 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:50:29,669 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:29,671 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691473829670, 0, 1, None]) gunicorn-web stdout | 2023-08-08 05:50:29,674 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:29,675 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:50:29,676 [197] [DEBUG] [app] Starting request: urn:request:1c8f69d1-dbae-4cb3-9d61-20fe267fbef1 (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:29,677 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,677 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,677 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:29,677 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:29,680 [199] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691473829679, 0, 1]) gunicorn-web stdout | 2023-08-08 05:50:29,681 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,681 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,683 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:50:29,684 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:29,684 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:29,684 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,684 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 05:50:29,685 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:29,685 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:29,685 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:50:29,685 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:29,685 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,685 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,685 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:29,686 [199] [DEBUG] [app] Ending request: urn:request:62a92b94-3297-4ca2-a0b2-a9e98a16dd12 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:62a92b94-3297-4ca2-a0b2-a9e98a16dd12', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:29,686 [197] [DEBUG] [app] Ending request: urn:request:1c8f69d1-dbae-4cb3-9d61-20fe267fbef1 (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:1c8f69d1-dbae-4cb3-9d61-20fe267fbef1', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:29,687 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:29,687 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:29,687 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:29 +0000] "GET /api/v1/user/ HTTP/1.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:29,687 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:29 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:29 +0000] "GET /api/v1/user/ HTTP/2.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.037 46 0.038) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:29 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 47 0.013) exportactionlogsworker stdout | 2023-08-08 05:50:29,770 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:50:29,831 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:50:29,858 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:50:29,866 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:50:29,867 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:50:29,871 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:30,019 [198] [DEBUG] [app] Starting request: urn:request:78986c29-3f0d-4fec-ab09-84b27c99474b (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:30,020 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,020 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,020 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:30,020 [197] [DEBUG] [app] Starting request: urn:request:ccda94d8-3aad-4875-9ac4-a4fc67c056c7 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:30,021 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,021 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,022 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:30,026 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:30,026 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:30,026 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,026 [198] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:50:30,026 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,027 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,027 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,027 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,027 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:30,028 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:30,028 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,028 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:50:30,028 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,028 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,028 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,028 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,028 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 05:50:30,029 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 2, 30, 28519), datetime.datetime(2023, 8, 8, 5, 50, 30, 28592)]) gunicorn-web stdout | 2023-08-08 05:50:30,030 [197] [DEBUG] [app] Ending request: urn:request:ccda94d8-3aad-4875-9ac4-a4fc67c056c7 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:ccda94d8-3aad-4875-9ac4-a4fc67c056c7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:30,031 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 62 0.012) gunicorn-web stdout | 2023-08-08 05:50:30,032 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:30,033 [198] [DEBUG] [app] Ending request: urn:request:78986c29-3f0d-4fec-ab09-84b27c99474b (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:78986c29-3f0d-4fec-ab09-84b27c99474b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:30,034 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 398 0.017) gunicorn-web stdout | 2023-08-08 05:50:30,034 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:30,037 [197] [DEBUG] [app] Starting request: urn:request:0afd9937-a55e-44ab-8d5e-fe42ad3fc6f7 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:30,038 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,038 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,038 [200] [DEBUG] [app] Starting request: urn:request:40fc0424-196a-4cfb-9f25-03baa0e8d0dd (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:30,038 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,038 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,038 [199] [DEBUG] [app] Starting request: urn:request:3a880be0-1ddd-43c2-955f-96a952e059d8 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:30,038 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:30,038 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,039 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,039 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:30,039 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:30,044 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:30,044 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:30,044 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,045 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:30,045 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:30,045 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,046 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:50:30,045 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:30,046 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,046 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:30,046 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,046 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,046 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,046 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,046 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:30,046 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:30,046 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,046 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,046 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:30,046 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:30,047 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:30,047 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:30,048 [199] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'build_test', 101]) gunicorn-web stdout | 2023-08-08 05:50:30,049 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:50:30,049 [200] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'whuquay', 101]) gunicorn-web stdout | 2023-08-08 05:50:30,050 [197] [DEBUG] [app] Ending request: urn:request:0afd9937-a55e-44ab-8d5e-fe42ad3fc6f7 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:0afd9937-a55e-44ab-8d5e-fe42ad3fc6f7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:30,051 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:30,051 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 84 0.015) gunicorn-web stdout | 2023-08-08 05:50:30,052 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:30,052 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:50:30,053 [199] [DEBUG] [app] Ending request: urn:request:3a880be0-1ddd-43c2-955f-96a952e059d8 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:3a880be0-1ddd-43c2-955f-96a952e059d8', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'false', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:30,054 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:30,054 [200] [DEBUG] [app] Ending request: urn:request:40fc0424-196a-4cfb-9f25-03baa0e8d0dd (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:40fc0424-196a-4cfb-9f25-03baa0e8d0dd', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'whuquay', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 114 0.017) gunicorn-web stdout | 2023-08-08 05:50:30,054 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:30,055 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 103 0.019) gunicorn-web stdout | 2023-08-08 05:50:30,055 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:30 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" globalpromstats stdout | 2023-08-08 05:50:30,338 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:50:30,931 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:50:31,179 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:50:31,467 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:50:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:50:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:42.293757+00:00 (in 9.999451 seconds) chunkcleanupworker stdout | 2023-08-08 05:50:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:42 UTC)" (scheduled at 2023-08-08 05:50:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:50:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:50:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 32, 294789), True, datetime.datetime(2023, 8, 8, 5, 50, 32, 294789), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:50:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:50:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:50:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:42 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:32,627 [199] [DEBUG] [app] Starting request: urn:request:a8f103a8-f9e3-4fa4-8e2a-31a0a97353bf (/api/v1/organization/build_test) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:32,627 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:32,627 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:32,628 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:32,634 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:32,634 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:32,634 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:32,635 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:32,636 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:32,637 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:32,637 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:32,637 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:32,639 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:50:32,641 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:32,642 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:32,642 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:50:32,642 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:32,642 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t2" ON ("t1"."role_id" = "t2"."id") WHERE ("t1"."organization_id" = %s)', [2]) gunicorn-web stdout | 2023-08-08 05:50:32,643 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:50:32,644 [199] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "repositorypermission" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [1]) gunicorn-web stdout | 2023-08-08 05:50:32,646 [199] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "teammember" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [1]) gunicorn-web stdout | 2023-08-08 05:50:32,646 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:32,646 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:32,646 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:32,647 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 05:50:32,648 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:32,650 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691473832649, 0, 2, None]) gunicorn-web stdout | 2023-08-08 05:50:32,653 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:32,654 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 05:50:32,655 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:32,657 [199] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691473832656, 0, 2]) gunicorn-web stdout | 2023-08-08 05:50:32,659 [199] [DEBUG] [app] Ending request: urn:request:a8f103a8-f9e3-4fa4-8e2a-31a0a97353bf (/api/v1/organization/build_test) {'endpoint': 'api.organization', 'request_id': 'urn:request:a8f103a8-f9e3-4fa4-8e2a-31a0a97353bf', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test', 'path': '/api/v1/organization/build_test', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:32,660 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:32 +0000] "GET /api/v1/organization/build_test HTTP/2.0" 200 770 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.035 483 0.034) gunicorn-web stdout | 2023-08-08 05:50:32,660 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:32 +0000] "GET /api/v1/organization/build_test HTTP/1.0" 200 770 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-registry stdout | 2023-08-08 05:50:32,866 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:50:32,873 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:50:32,877 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:50:32,881 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:50:32,883 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:50:32,889 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:50:32,892 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:50:32,895 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:50:32,899 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-web stdout | 2023-08-08 05:50:33,309 [197] [DEBUG] [app] Starting request: urn:request:a28735a6-9b6b-488f-a2c0-bd4a5f9ae22f (/api/v1/organization/build_test/quota) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:33,309 [200] [DEBUG] [app] Starting request: urn:request:951baf27-3bda-427e-a0cf-f405a4feefc4 (/api/v1/organization/build_test/proxycache) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:33,309 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,309 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,309 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,309 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,310 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:33,310 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:33,317 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:33,317 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:33,317 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:33,317 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,317 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:33,317 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,317 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,317 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,317 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,317 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,317 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,317 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,317 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,317 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,318 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:50:33,319 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:50:33,321 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,321 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,321 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,321 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,321 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,321 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,321 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:33,321 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:33,322 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', True, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:33,322 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:33,324 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 05:50:33,324 [200] [DEBUG] [app] Ending request: urn:request:951baf27-3bda-427e-a0cf-f405a4feefc4 (/api/v1/organization/build_test/proxycache) {'endpoint': 'api.organizationproxycacheconfig', 'request_id': 'urn:request:951baf27-3bda-427e-a0cf-f405a4feefc4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/proxycache', 'path': '/api/v1/organization/build_test/proxycache', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:33,325 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:33,325 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."quota_id", "t1"."quota_type_id", "t1"."percent_of_limit" FROM "quotalimits" AS "t1" INNER JOIN "quotatype" AS "t2" ON ("t1"."quota_type_id" = "t2"."id") WHERE ("t1"."quota_id" = %s)', [None]) gunicorn-web stdout | 2023-08-08 05:50:33,325 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:33 +0000] "GET /api/v1/organization/build_test/proxycache HTTP/1.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:33 +0000] "GET /api/v1/organization/build_test/proxycache HTTP/2.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 322 0.019) gunicorn-web stdout | 2023-08-08 05:50:33,327 [197] [DEBUG] [app] Ending request: urn:request:a28735a6-9b6b-488f-a2c0-bd4a5f9ae22f (/api/v1/organization/build_test/quota) {'endpoint': 'api.organizationquotalist', 'request_id': 'urn:request:a28735a6-9b6b-488f-a2c0-bd4a5f9ae22f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/quota', 'path': '/api/v1/organization/build_test/quota', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:33,328 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:33,328 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:33 +0000] "GET /api/v1/organization/build_test/quota HTTP/1.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:33 +0000] "GET /api/v1/organization/build_test/quota HTTP/2.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 406 0.021) gunicorn-web stdout | 2023-08-08 05:50:33,510 [199] [DEBUG] [app] Starting request: urn:request:9d768396-0c28-4130-9365-eee84f1eb9e4 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:33,510 [197] [DEBUG] [app] Starting request: urn:request:1e036d53-1efb-4f5f-98bc-4a8a69186e0c (/api/v1/organization/build_test/prototypes) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:33,510 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,511 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,511 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,511 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,511 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:33,511 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:33,518 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:33,519 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:33,519 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,519 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:33,519 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:33,519 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,519 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,519 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,519 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,519 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,519 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:33,519 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:33,519 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,519 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,519 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,520 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:33,521 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:50:33,522 [199] [DEBUG] [peewee] ('(SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t1"."visibility_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") WHERE ((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'build_test', 1, 3, 1, 'build_test', 101]) gunicorn-web stdout | 2023-08-08 05:50:33,523 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,524 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,524 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:50:33,524 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:33,524 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:33,526 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:50:33,526 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" LEFT OUTER JOIN "user" AS "t2" ON ("t2"."id" = "t1"."activating_user_id") LEFT OUTER JOIN "user" AS "t3" ON ("t3"."id" = "t1"."delegate_user_id") LEFT OUTER JOIN "team" AS "t4" ON ("t4"."id" = "t1"."delegate_team_id") LEFT OUTER JOIN "role" AS "t5" ON ("t5"."id" = "t1"."role_id") WHERE ("t1"."org_id" = %s)', [2]) gunicorn-web stdout | 2023-08-08 05:50:33,528 [199] [DEBUG] [app] Ending request: urn:request:9d768396-0c28-4130-9365-eee84f1eb9e4 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:9d768396-0c28-4130-9365-eee84f1eb9e4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'true', 'quota': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:33,528 [197] [DEBUG] [app] Ending request: urn:request:1e036d53-1efb-4f5f-98bc-4a8a69186e0c (/api/v1/organization/build_test/prototypes) {'endpoint': 'api.permissionprototypelist', 'request_id': 'urn:request:1e036d53-1efb-4f5f-98bc-4a8a69186e0c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/prototypes', 'path': '/api/v1/organization/build_test/prototypes', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:33,529 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:33,529 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:33,529 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:33 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:33 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 106 0.020) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:33 +0000] "GET /api/v1/organization/build_test/prototypes HTTP/2.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 66 0.021) gunicorn-web stdout | 2023-08-08 05:50:33,530 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:33 +0000] "GET /api/v1/organization/build_test/prototypes HTTP/1.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 05:50:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:50:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:45.371682+00:00 (in 9.999429 seconds) notificationworker stdout | 2023-08-08 05:50:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:45 UTC)" (scheduled at 2023-08-08 05:50:35.371682+00:00) notificationworker stdout | 2023-08-08 05:50:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:50:35,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 35, 372897), True, datetime.datetime(2023, 8, 8, 5, 50, 35, 372897), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:50:35,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:50:35,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:50:35,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:50:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:50:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:54.393627+00:00 (in 17.813271 seconds) namespacegcworker stdout | 2023-08-08 05:50:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:36 UTC)" (scheduled at 2023-08-08 05:50:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:50:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:50:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:37,179 [197] [DEBUG] [app] Starting request: urn:request:38891649-80ea-4c18-b572-ffc9503f3e34 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:50:37,180 [200] [DEBUG] [app] Starting request: urn:request:10650d10-bcda-4511-9565-f381e8a7f3a1 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:50:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:50:37,185 [210] [DEBUG] [app] Starting request: urn:request:99077cfa-92c6-402a-8eef-244cfe20f6c5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:50:37,186 [210] [DEBUG] [app] Ending request: urn:request:99077cfa-92c6-402a-8eef-244cfe20f6c5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:99077cfa-92c6-402a-8eef-244cfe20f6c5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:50:37,186 [209] [DEBUG] [app] Starting request: urn:request:706c9776-219f-4dee-9779-9246958cf034 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:50:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:50:37,186 [209] [DEBUG] [app] Ending request: urn:request:706c9776-219f-4dee-9779-9246958cf034 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:706c9776-219f-4dee-9779-9246958cf034', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:50:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:50:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 05:50:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:37,191 [199] [DEBUG] [app] Starting request: urn:request:7ba2e977-cd49-4658-a133-5c491fb63ca3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:50:37,192 [199] [DEBUG] [app] Ending request: urn:request:7ba2e977-cd49-4658-a133-5c491fb63ca3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7ba2e977-cd49-4658-a133-5c491fb63ca3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:50:37,191 [198] [DEBUG] [app] Starting request: urn:request:e14e40ff-b160-4dd8-bacb-48cca395c08f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:50:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:37,192 [198] [DEBUG] [app] Ending request: urn:request:e14e40ff-b160-4dd8-bacb-48cca395c08f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e14e40ff-b160-4dd8-bacb-48cca395c08f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:50:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:50:37,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:50:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:50:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:50:37,198 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:50:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:07.197412+00:00 (in 29.999206 seconds) buildlogsarchiver stdout | 2023-08-08 05:50:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:07 UTC)" (scheduled at 2023-08-08 05:50:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:50:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 50, 37, 198729), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:37,202 [197] [DEBUG] [app] Ending request: urn:request:38891649-80ea-4c18-b572-ffc9503f3e34 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:38891649-80ea-4c18-b572-ffc9503f3e34', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:50:37,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:50:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:50:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 05:50:37,205 [200] [DEBUG] [app] Ending request: urn:request:10650d10-bcda-4511-9565-f381e8a7f3a1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:10650d10-bcda-4511-9565-f381e8a7f3a1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:50:37,205 [200] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:50:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:50:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:50:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:37,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:50:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:50:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gcworker stdout | 2023-08-08 05:50:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:50:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:49.417660+00:00 (in 9.999269 seconds) gcworker stdout | 2023-08-08 05:50:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:49 UTC)" (scheduled at 2023-08-08 05:50:39.417660+00:00) gcworker stdout | 2023-08-08 05:50:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264239419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:50:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:50:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:50:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:49 UTC)" executed successfully builder stdout | 2023-08-08 05:50:39,881 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:50:39,891 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:50:39,891 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:50:39,896 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:50:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:50:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:52.293757+00:00 (in 9.999456 seconds) chunkcleanupworker stdout | 2023-08-08 05:50:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:52 UTC)" (scheduled at 2023-08-08 05:50:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:50:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:50:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 42, 294700), True, datetime.datetime(2023, 8, 8, 5, 50, 42, 294700), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:50:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:50:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:50:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:50:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:50:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:55.371682+00:00 (in 9.999357 seconds) notificationworker stdout | 2023-08-08 05:50:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:55 UTC)" (scheduled at 2023-08-08 05:50:45.371682+00:00) notificationworker stdout | 2023-08-08 05:50:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:50:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 45, 372994), True, datetime.datetime(2023, 8, 8, 5, 50, 45, 372994), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:50:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:50:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:50:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:55 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:46,538 [199] [DEBUG] [app] Starting request: urn:request:4cfa32b4-9955-4adb-80b1-a9a7ebd95a68 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:46,538 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,538 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,539 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,545 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:46,545 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:46,545 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,546 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,547 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,547 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,547 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,548 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:50:46,551 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,551 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,551 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,551 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:46,552 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,555 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,557 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,559 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:50:46,560 [199] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [2, 'github', 2, '', '245e22c3-ba4f-4e53-9a58-fce8fb4eb90f', 1, False, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,562 [199] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [2, 0, datetime.date(2023, 8, 7)]) gunicorn-web stdout | 2023-08-08 05:50:46,563 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [2, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,564 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,565 [199] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [1, 2, 1]) gunicorn-web stdout | 2023-08-08 05:50:46,569 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_user_id" IS %s))', [2, 1, None, None]) gunicorn-web stdout | 2023-08-08 05:50:46,571 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_team_id" IS %s))', [2, 1, None, None]) gunicorn-web stdout | 2023-08-08 05:50:46,573 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,575 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,576 [199] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [14, 2, 1, 2, datetime.datetime(2023, 8, 8, 5, 50, 46, 574605), '10.130.2.1', '{"repo": "github", "namespace": "build_test"}']) gunicorn-web stdout | 2023-08-08 05:50:46,581 [199] [DEBUG] [app] Ending request: urn:request:4cfa32b4-9955-4adb-80b1-a9a7ebd95a68 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:4cfa32b4-9955-4adb-80b1-a9a7ebd95a68', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository', 'path': '/api/v1/repository', 'parameters': {}, 'json_body': {'namespace': 'build_test', 'repository': 'github', 'visibility': 'private', 'description': '', 'repo_kind': 'image'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:46,582 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:46 +0000] "POST /api/v1/repository HTTP/2.0" 201 63 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.046 1038 0.045) gunicorn-web stdout | 2023-08-08 05:50:46,582 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:46 +0000] "POST /api/v1/repository HTTP/1.0" 201 63 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 05:50:46,932 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gunicorn-web stdout | 2023-08-08 05:50:46,952 [199] [DEBUG] [app] Starting request: urn:request:34869dfa-6415-4723-9cf7-3da45fece7cf (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:46,952 [197] [DEBUG] [app] Starting request: urn:request:7fce7f6a-3012-458c-85db-252078c12301 (/api/v1/repository/build_test/github/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:46,952 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,952 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,952 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,952 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,953 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,953 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,960 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:46,960 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:46,960 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:46,960 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:46,960 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,960 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,960 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:46,960 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,960 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,960 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:46,960 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,960 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,960 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,961 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,961 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,961 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,963 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:46,963 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:46,969 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,969 [199] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:46,969 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:50:46,970 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,971 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,972 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [2, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,973 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [2, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:50:46,974 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,975 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,975 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:46,976 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [2, None, 1691473846975, False, 101, 0]) gunicorn-web stdout | 2023-08-08 05:50:46,976 [199] [DEBUG] [app] Ending request: urn:request:34869dfa-6415-4723-9cf7-3da45fece7cf (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:34869dfa-6415-4723-9cf7-3da45fece7cf', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:46,976 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:46,977 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:46 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:46 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/2.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 88 0.027) gunicorn-web stdout | 2023-08-08 05:50:46,978 [197] [DEBUG] [app] Ending request: urn:request:7fce7f6a-3012-458c-85db-252078c12301 (/api/v1/repository/build_test/github/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:7fce7f6a-3012-458c-85db-252078c12301', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/github/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:46,979 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:46 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 536 0.030) gunicorn-web stdout | 2023-08-08 05:50:46,979 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:46 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" exportactionlogsworker stdout | 2023-08-08 05:50:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:50:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:25.514341+00:00 (in 38.115235 seconds) exportactionlogsworker stdout | 2023-08-08 05:50:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:47 UTC)" (scheduled at 2023-08-08 05:50:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:50:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:50:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 47, 399486), True, datetime.datetime(2023, 8, 8, 5, 50, 47, 399486), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:50:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:50:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:50:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:47 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:47,732 [199] [DEBUG] [app] Starting request: urn:request:583f88dc-030f-4422-819b-d3b088d4ac7a (/repository/build_test/github/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:47,733 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:47,742 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:47,744 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:47,747 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['github', 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:47,753 [199] [DEBUG] [app] Ending request: urn:request:583f88dc-030f-4422-819b-d3b088d4ac7a (/repository/build_test/github/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:583f88dc-030f-4422-819b-d3b088d4ac7a', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f', 'path': '/repository/build_test/github/status', 'parameters': {'token': '245e22c3-ba4f-4e53-9a58-fce8fb4eb90f'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:47,754 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:47,754 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:47 +0000] "GET /repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:47 +0000] "GET /repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.024 124 0.024) gunicorn-web stdout | 2023-08-08 05:50:47,884 [198] [DEBUG] [app] Starting request: urn:request:e2abd476-fb83-4841-b55d-a61bc6cb1727 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:47,884 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:47,884 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:47,885 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:47,891 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:47,892 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:47,892 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:47,892 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:47,892 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:47,892 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:47,892 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:47,892 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:47,894 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:47,900 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:50:47,902 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:47,905 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:50:47,909 [198] [DEBUG] [app] Ending request: urn:request:e2abd476-fb83-4841-b55d-a61bc6cb1727 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:e2abd476-fb83-4841-b55d-a61bc6cb1727', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:47,910 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:47,910 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:47 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:47 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 73 0.028) logrotateworker stdout | 2023-08-08 05:50:48,119 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:50:49,338 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:50:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:50:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:59.417660+00:00 (in 9.999472 seconds) gcworker stdout | 2023-08-08 05:50:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:59 UTC)" (scheduled at 2023-08-08 05:50:49.417660+00:00) gcworker stdout | 2023-08-08 05:50:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264249418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:50:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:50:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:50:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:50:59 UTC)" executed successfully securityworker stdout | 2023-08-08 05:50:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:50:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:50:56.890741+00:00 (in 7.202686 seconds) securityworker stdout | 2023-08-08 05:50:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:19 UTC)" (scheduled at 2023-08-08 05:50:49.687363+00:00) securityworker stdout | 2023-08-08 05:50:49,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:50:49,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:50:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:50:49,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:50:49,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:50:49,700 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:50:49,700 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:19 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:50:49,749 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:50:49,906 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:50:49,914 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:50:49,914 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:50:49,919 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:50,669 [200] [DEBUG] [app] Starting request: urn:request:8d536796-1ddc-4101-ad19-9d60852fc548 (/api/v1/repository/build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:50,669 [199] [DEBUG] [app] Starting request: urn:request:564ad20b-368b-4533-9e91-0a2e35a411f2 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:50,669 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:50,669 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:50,669 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:50,669 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:50,670 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:50,670 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:50,678 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:50,678 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:50,678 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:50,678 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:50,678 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:50,678 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:50,678 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:50,678 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:50,678 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:50,678 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:50,678 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:50,678 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:50,678 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:50,678 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:50,678 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:50,678 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:50,681 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:50,681 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:50,687 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:50:50,688 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:50:50,689 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:50,689 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:50,691 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'github']) gunicorn-web stdout | 2023-08-08 05:50:50,692 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 10]) gunicorn-web stdout | 2023-08-08 05:50:50,694 [200] [DEBUG] [app] Ending request: urn:request:8d536796-1ddc-4101-ad19-9d60852fc548 (/api/v1/repository/build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:8d536796-1ddc-4101-ad19-9d60852fc548', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/', 'path': '/api/v1/repository/build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:50,694 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:50,695 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:50 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:50 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 505 0.027) gunicorn-web stdout | 2023-08-08 05:50:50,696 [199] [DEBUG] [app] Ending request: urn:request:564ad20b-368b-4533-9e91-0a2e35a411f2 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:564ad20b-368b-4533-9e91-0a2e35a411f2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=10', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:50,697 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:50 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 73 0.029) gunicorn-web stdout | 2023-08-08 05:50:50,697 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:50 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:52,179 [198] [DEBUG] [app] Starting request: urn:request:1763c1a9-f5fe-4170-af39-e380f23b2e71 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:50:52,179 [199] [DEBUG] [app] Starting request: urn:request:0ecd61cb-1e6e-4e0a-949a-ad26a718031a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:50:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:52,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:50:52,186 [203] [DEBUG] [app] Starting request: urn:request:2e2c3f90-c8df-4b73-8ef6-dfe49ce69380 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:50:52,186 [210] [DEBUG] [app] Starting request: urn:request:8e90d395-23e5-4391-8407-091003b84b47 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:50:52,186 [203] [DEBUG] [app] Ending request: urn:request:2e2c3f90-c8df-4b73-8ef6-dfe49ce69380 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2e2c3f90-c8df-4b73-8ef6-dfe49ce69380', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:50:52,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:50:52,187 [210] [DEBUG] [app] Ending request: urn:request:8e90d395-23e5-4391-8407-091003b84b47 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8e90d395-23e5-4391-8407-091003b84b47', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 05:50:52,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:50:52,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:52,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:52,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:52,189 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:50:52,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:52,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:50:52,193 [197] [DEBUG] [app] Starting request: urn:request:901e01d7-951f-40c9-aa73-10dacd0a783d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:50:52,193 [197] [DEBUG] [app] Ending request: urn:request:901e01d7-951f-40c9-aa73-10dacd0a783d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:901e01d7-951f-40c9-aa73-10dacd0a783d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:50:52,193 [200] [DEBUG] [app] Starting request: urn:request:120c3ab7-279c-443f-b29d-67aa397500cc (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:50:52,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:50:52,194 [200] [DEBUG] [app] Ending request: urn:request:120c3ab7-279c-443f-b29d-67aa397500cc (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:120c3ab7-279c-443f-b29d-67aa397500cc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:50:52,194 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:52,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:50:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:50:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:50:52,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:50:52,194 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:50:52,195 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:50:52,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:50:52,204 [198] [DEBUG] [app] Ending request: urn:request:1763c1a9-f5fe-4170-af39-e380f23b2e71 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1763c1a9-f5fe-4170-af39-e380f23b2e71', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:50:52,205 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:50:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:50:52,205 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:50:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:50:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:50:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:02.293757+00:00 (in 9.999429 seconds) chunkcleanupworker stdout | 2023-08-08 05:50:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:02 UTC)" (scheduled at 2023-08-08 05:50:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:50:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:50:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 52, 294797), True, datetime.datetime(2023, 8, 8, 5, 50, 52, 294797), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:50:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:50:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:50:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:02 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:52,322 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:50:52,328 [199] [DEBUG] [app] Ending request: urn:request:0ecd61cb-1e6e-4e0a-949a-ad26a718031a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0ecd61cb-1e6e-4e0a-949a-ad26a718031a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:50:52,328 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:52,328 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:50:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:50:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.150 47 0.150) gcworker stdout | 2023-08-08 05:50:52,555 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:50:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:50:54,285 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:13.097375+00:00 (in 18.812403 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:50:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:54 UTC)" (scheduled at 2023-08-08 05:50:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:50:54,286 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:50:54,286 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:50:54,354 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} namespacegcworker stdout | 2023-08-08 05:50:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:50:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:36.579841+00:00 (in 42.185606 seconds) namespacegcworker stdout | 2023-08-08 05:50:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:54 UTC)" (scheduled at 2023-08-08 05:50:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:50:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:50:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 54, 394898), True, datetime.datetime(2023, 8, 8, 5, 50, 54, 394898), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:50:54,405 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:50:54,405 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:50:54,405 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:51:54 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:50:54,430 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:50:54,682 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:50:54,686 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:50:54,689 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:50:54,694 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:50:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:50:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:02.178530+00:00 (in 6.805820 seconds) notificationworker stdout | 2023-08-08 05:50:55,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:05 UTC)" (scheduled at 2023-08-08 05:50:55.371682+00:00) notificationworker stdout | 2023-08-08 05:50:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:50:55,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 50, 55, 373271), True, datetime.datetime(2023, 8, 8, 5, 50, 55, 373271), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:50:55,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:50:55,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:50:55,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:50:55,791 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:50:56,356 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:50:56,546 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 05:50:56,882 [200] [DEBUG] [app] Starting request: urn:request:1a33cf9c-e50a-4c02-980e-b8e94cd589df (/oauth2/github/callback/trigger/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:56,882 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:56,882 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:56,883 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:56,890 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:56,890 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:56,890 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:56,890 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:56,890 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:56,890 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') securityworker stdout | 2023-08-08 05:50:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:50:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:19.687363+00:00 (in 22.796074 seconds) securityworker stdout | 2023-08-08 05:50:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:26 UTC)" (scheduled at 2023-08-08 05:50:56.890741+00:00) securityworker stdout | 2023-08-08 05:50:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:50:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:50:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:50:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:50:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST gunicorn-web stdout | 2023-08-08 05:50:56,893 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) securityworker stdout | 2023-08-08 05:50:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:50:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:50:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:50:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:50:56,899 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') securityworker stdout | 2023-08-08 05:50:56,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST gunicorn-web stdout | 2023-08-08 05:50:56,903 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443 securityworker stdout | 2023-08-08 05:50:56,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:50:56,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:50:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:50:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:50:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:50:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:50:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:50:56,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:50:57,045 [200] [DEBUG] [urllib3.connectionpool] https://github.com:443 "POST /login/oauth/access_token?code=3acfac391c641b4a1db7&grant_type=authorization_code&redirect_uri=https%3A%2F%2Fquayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com%2Foauth2%2Fgithub%2Fcallback&client_id=2ddf9640aa6b5e341a7d&client_secret=3764a2acebe77c5406143cb606e79683e994ae38 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:50:57,047 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:57,049 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:57,050 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "buildtriggerservice" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['github', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:57,052 [200] [DEBUG] [peewee] ('INSERT INTO "repositorybuildtrigger" ("uuid", "service_id", "repository_id", "connected_user_id", "secure_auth_token", "fully_migrated", "config", "pull_robot_id", "enabled", "disabled_datetime", "successive_failure_count", "successive_internal_error_count") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repositorybuildtrigger"."id"', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 2, 1, 'v0$$ZXz9fg6IfOmmIjqfVObhiTlHx0Fdkz+KTywlswvlUP5eVk7rIlIqy/AF6QFZwc7GZWA467JrklqJMFKY0e+QMc3DqcQT', False, '{}', None, True, datetime.datetime(2023, 8, 8, 5, 50, 57, 51804), 0, 0]) gunicorn-web stdout | 2023-08-08 05:50:57,057 [200] [DEBUG] [endpoints.githubtrigger] Redirecting to full url: /repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e gunicorn-web stdout | 2023-08-08 05:50:57,058 [200] [DEBUG] [app] Ending request: urn:request:1a33cf9c-e50a-4c02-980e-b8e94cd589df (/oauth2/github/callback/trigger/build_test/github) {'endpoint': 'callback.attach_github_build_trigger', 'request_id': 'urn:request:1a33cf9c-e50a-4c02-980e-b8e94cd589df', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/oauth2/github/callback/trigger/build_test/github?code=3acfac391c641b4a1db7', 'path': '/oauth2/github/callback/trigger/build_test/github', 'parameters': {'code': '3acfac391c641b4a1db7'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:57,059 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:57 +0000] "GET /oauth2/github/callback/trigger/build_test/github?code=3acfac391c641b4a1db7 HTTP/2.0" 302 355 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.179 847 0.179) gunicorn-web stdout | 2023-08-08 05:50:57,059 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:57 +0000] "GET /oauth2/github/callback/trigger/build_test/github?code=3acfac391c641b4a1db7 HTTP/1.0" 302 355 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:57,076 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:50:57,082 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:50:57,087 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:50:57,091 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:50:57,093 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 05:50:57,377 [199] [DEBUG] [app] Starting request: urn:request:80c4875e-9411-4ed5-a958-f15c43e4a31a (/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:57,386 [199] [DEBUG] [endpoints.api.discovery] Unable to find method for POST in class gunicorn-web stdout | 2023-08-08 05:50:57,391 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:57,411 [199] [DEBUG] [app] Ending request: urn:request:80c4875e-9411-4ed5-a958-f15c43e4a31a (/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'endpoint': 'web.buildtrigger', 'request_id': 'urn:request:80c4875e-9411-4ed5-a958-f15c43e4a31a', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'path': '/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:57,412 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:57 +0000] "GET /repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/1.0" 200 85192 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:57 +0000] "GET /repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/2.0" 200 13315 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.037 434 0.037) buildlogsarchiver stdout | 2023-08-08 05:50:57,754 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:50:57,854 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:58 +0000] "GET /static/img/quay_favicon.png HTTP/2.0" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 214 -) gunicorn-web stdout | 2023-08-08 05:50:58,283 [199] [DEBUG] [app] Starting request: urn:request:bd6eb4e3-6204-4fe9-8203-223c5a2f7c4b (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:58,283 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,283 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,284 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,290 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:58,291 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:58,291 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,291 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,291 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,291 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,291 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,292 [199] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 05:50:58,295 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,295 [199] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 05:50:58,297 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:50:58,298 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,299 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:50:58,301 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,303 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691473858302, 0, 1, None]) gunicorn-web stdout | 2023-08-08 05:50:58,306 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,307 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:50:58,308 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,309 [197] [DEBUG] [app] Starting request: urn:request:52e76635-7047-4c92-9fb4-6eb4a62200d4 (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:58,309 [199] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691473858309, 0, 1]) gunicorn-web stdout | 2023-08-08 05:50:58,309 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,310 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,310 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,310 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,311 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,312 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:50:58,313 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,313 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,314 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,314 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:50:58,314 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,314 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,314 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,314 [199] [DEBUG] [app] Ending request: urn:request:bd6eb4e3-6204-4fe9-8203-223c5a2f7c4b (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:bd6eb4e3-6204-4fe9-8203-223c5a2f7c4b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:58,315 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/user/ HTTP/2.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.035 171 0.034) gunicorn-web stdout | 2023-08-08 05:50:58,316 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/user/ HTTP/1.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:58,317 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:58,317 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:58,317 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,318 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 05:50:58,320 [197] [DEBUG] [app] Ending request: urn:request:52e76635-7047-4c92-9fb4-6eb4a62200d4 (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:52e76635-7047-4c92-9fb4-6eb4a62200d4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:58,320 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 47 0.013) gunicorn-web stdout | 2023-08-08 05:50:58,321 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:58,394 [199] [DEBUG] [app] Starting request: urn:request:2361480b-6eac-4c5d-8892-50531ef1e6aa (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:58,394 [197] [DEBUG] [app] Starting request: urn:request:6eb59bb1-8609-4f8f-bab3-eba8484a6d2b (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:58,394 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,394 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,394 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,394 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,395 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,395 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,401 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:58,402 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:58,402 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:58,402 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,402 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:58,402 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,402 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:58,402 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,402 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:58,402 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,402 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,402 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,402 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,402 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,402 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,402 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,404 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:58,404 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:58,411 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,411 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,411 [199] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:58,412 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,412 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,413 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [2, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,415 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [2, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:50:58,415 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,416 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,416 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,417 [199] [DEBUG] [app] Ending request: urn:request:2361480b-6eac-4c5d-8892-50531ef1e6aa (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:2361480b-6eac-4c5d-8892-50531ef1e6aa', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github?includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:58,418 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/2.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 75 0.026) gunicorn-web stdout | 2023-08-08 05:50:58,418 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/1.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:58,418 [197] [DEBUG] [app] Ending request: urn:request:6eb59bb1-8609-4f8f-bab3-eba8484a6d2b (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:6eb59bb1-8609-4f8f-bab3-eba8484a6d2b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:58,419 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/2.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 94 0.028) gunicorn-web stdout | 2023-08-08 05:50:58,419 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/1.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:58,646 [200] [DEBUG] [app] Starting request: urn:request:ead8c3ee-dc14-49c3-9be9-ac85f37dd0d6 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:58,646 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,646 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,647 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,647 [197] [DEBUG] [app] Starting request: urn:request:088fd8ce-b657-4f77-be2d-469d51cd68a4 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:58,648 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,648 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,649 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,654 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:58,654 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:58,654 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,654 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:50:58,654 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,654 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,654 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,654 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,655 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:58,655 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:58,655 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,655 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:50:58,655 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,655 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,655 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,655 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,656 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 2, 58, 655905), datetime.datetime(2023, 8, 8, 5, 50, 58, 655958)]) gunicorn-web stdout | 2023-08-08 05:50:58,656 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 05:50:58,657 [197] [DEBUG] [app] Ending request: urn:request:088fd8ce-b657-4f77-be2d-469d51cd68a4 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:088fd8ce-b657-4f77-be2d-469d51cd68a4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:58,658 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 62 0.012) gunicorn-web stdout | 2023-08-08 05:50:58,659 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:58,660 [200] [DEBUG] [app] Ending request: urn:request:ead8c3ee-dc14-49c3-9be9-ac85f37dd0d6 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:ead8c3ee-dc14-49c3-9be9-ac85f37dd0d6', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:58,661 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 399 0.017) gunicorn-web stdout | 2023-08-08 05:50:58,662 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:58 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:50:58,744 [198] [DEBUG] [app] Starting request: urn:request:d62ca1e8-5805-4fca-b2d2-9ce8d5e7b1b2 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:50:58,744 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,744 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,745 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,751 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:50:58,751 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:50:58,751 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,751 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:50:58,751 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,751 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,751 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,751 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,753 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:50:58,759 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:50:58,760 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,763 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,766 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:50:58,767 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:50:58,770 [198] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:50:58,913 [198] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:50:58,913 [198] [DEBUG] [github.Requester] GET https://api.github.com/user {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:50:58 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"9cb1ef85201d07db343ca73c6767340ac0fc5fb90c826222a744bc0bb0ecfe2b"', 'last-modified': 'Fri, 04 Aug 2023 13:02:06 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4999', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '1', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '5D71:67D0:D88341:1BD0405:64D1D7C2'} {"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false,"name":"Hu Weihua","company":null,"blog":"","location":null,"email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":15,"public_gists":0,"followers":5,"following":0,"created_at":"2014-04-22T08:41:09Z","updated_at":"2023-08-04T13:02:06Z"} securityworker stdout | 2023-08-08 05:50:58,982 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gunicorn-web stdout | 2023-08-08 05:50:59,002 [198] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user/orgs HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:50:59,003 [198] [DEBUG] [github.Requester] GET https://api.github.com/user/orgs {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:50:58 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"d13b5efb65207c14dcee4455a3f1eaf0a71ec38da08cb11bc838e0fe4536cdf8"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:org, read:org, repo, user, write:org', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4998', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '2', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '5D71:67D0:D8835E:1BD0440:64D1D7C2'} [{"login":"finley","id":422903,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQyMjkwMw==","url":"https://api.github.com/orgs/finley","repos_url":"https://api.github.com/orgs/finley/repos","events_url":"https://api.github.com/orgs/finley/events","hooks_url":"https://api.github.com/orgs/finley/hooks","issues_url":"https://api.github.com/orgs/finley/issues","members_url":"https://api.github.com/orgs/finley/members{/member}","public_members_url":"https://api.github.com/orgs/finley/public_members{/member}","avatar_url":"https://avatars.githubusercontent.com/u/422903?v=4","description":null},{"login":"xcat2","id":10124414,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwMTI0NDE0","url":"https://api.github.com/orgs/xcat2","repos_url":"https://api.github.com/orgs/xcat2/repos","events_url":"https://api.github.com/orgs/xcat2/events","hooks_url":"https://api.github.com/orgs/xcat2/hooks","issues_url":"https://api.github.com/orgs/xcat2/issues","members_url":"https://api.github.com/orgs/xcat2/members{/member}","public_members_url":"https://api.github.com/orgs/xcat2/public_members{/member}","avatar_url":"https://avatars.githubusercontent.com/u/10124414?v=4","description":" xCAT offers complete management for HPC clusters, RenderFarms, Grids, WebFarms, Online Gaming Infrastructure, Clouds, Datacenters"}] gunicorn-web stdout | 2023-08-08 05:50:59,007 [198] [DEBUG] [app] Ending request: urn:request:d62ca1e8-5805-4fca-b2d2-9ce8d5e7b1b2 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces) {'endpoint': 'api.buildtriggersourcenamespaces', 'request_id': 'urn:request:d62ca1e8-5805-4fca-b2d2-9ce8d5e7b1b2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:50:59,007 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:50:59,008 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:50:59 +0000] "GET /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces HTTP/1.0" 200 492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:50:59 +0000] "GET /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces HTTP/2.0" 200 492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.266 101 0.266) gcworker stdout | 2023-08-08 05:50:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:50:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:09.417660+00:00 (in 9.999455 seconds) gcworker stdout | 2023-08-08 05:50:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:09 UTC)" (scheduled at 2023-08-08 05:50:59.417660+00:00) gcworker stdout | 2023-08-08 05:50:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264259418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:50:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:50:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:50:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:50:59,810 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:50:59,868 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:50:59,925 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:50:59,934 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:50:59,935 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:50:59,939 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:51:00,375 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:51:00,943 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-web stdout | 2023-08-08 05:51:01,176 [199] [DEBUG] [app] Starting request: urn:request:f9b5929b-a2d5-4949-9fa5-d20cf789fb85 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/sources) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:51:01,176 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:01,176 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:01,177 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:01,183 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:51:01,184 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:51:01,184 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:01,184 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:51:01,184 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:01,184 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:01,184 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:01,184 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:01,187 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:51:01,193 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:51:01,194 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:01,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:01,200 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:01,203 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:01,204 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:01,206 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 namespacegcworker stdout | 2023-08-08 05:51:01,217 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-web stdout | 2023-08-08 05:51:01,366 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:01,367 [199] [DEBUG] [github.Requester] GET https://api.github.com/user {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:01 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"9cb1ef85201d07db343ca73c6767340ac0fc5fb90c826222a744bc0bb0ecfe2b"', 'last-modified': 'Fri, 04 Aug 2023 13:02:06 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4997', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '3', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E94E:20D2:D85980:1BC5C17:64D1D7C5'} {"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false,"name":"Hu Weihua","company":null,"blog":"","location":null,"email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":15,"public_gists":0,"followers":5,"following":0,"created_at":"2014-04-22T08:41:09Z","updated_at":"2023-08-04T13:02:06Z"} expiredappspecifictokenworker stdout | 2023-08-08 05:51:01,506 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-web stdout | 2023-08-08 05:51:02,025 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user/repos?type=owner&sort=updated HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:02,026 [199] [DEBUG] [github.Requester] GET https://api.github.com/user/repos?type=owner&sort=updated {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:02 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"4789a60480942469b3418501dba21b62e89232cb34f25e54b0ac0633df4fd700"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4996', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '4', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E94E:20D2:D859B5:1BC5C69:64D1D7C5'} [{"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":473030298,"node_id":"R_kgDOHDHemg","name":"quay_build_test_repo2","full_name":"hu-weihua/quay_build_test_repo2","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test_repo2","description":"quay_build_test_repo2","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/deployments","created_at":"2022-03-23T04:00:22Z","updated_at":"2022-03-23T04:01:50Z","pushed_at":"2022-03-23T04:01:47Z","git_url":"git://github.com/hu-weihua/quay_build_test_repo2.git","ssh_url":"git@github.com:hu-weihua/quay_build_test_repo2.git","clone_url":"https://github.com/hu-weihua/quay_build_test_repo2.git","svn_url":"https://github.com/hu-weihua/quay_build_test_repo2","homepage":null,"size":1,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":473864393,"node_id":"R_kgDOHD6YyQ","name":"node-app","full_name":"hu-weihua/node-app","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/node-app","description":null,"fork":true,"url":"https://api.github.com/repos/hu-weihua/node-app","forks_url":"https://api.github.com/repos/hu-weihua/node-app/forks","keys_url":"https://api.github.com/repos/hu-weihua/node-app/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/node-app/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/node-app/teams","hooks_url":"https://api.github.com/repos/hu-weihua/node-app/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/node-app/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/node-app/events","assignees_url":"https://api.github.com/repos/hu-weihua/node-app/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/node-app/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/node-app/tags","blobs_url":"https://api.github.com/repos/hu-weihua/node-app/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/node-app/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/node-app/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/node-app/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/node-app/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/node-app/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/node-app/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/node-app/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/node-app/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/node-app/subscription","commits_url":"https://api.github.com/repos/hu-weihua/node-app/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/node-app/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/node-app/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/node-app/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/node-app/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/node-app/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/node-app/merges","archive_url":"https://api.github.com/repos/hu-weihua/node-app/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/node-app/downloads","issues_url":"https://api.github.com/repos/hu-weihua/node-app/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/node-app/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/node-app/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/node-app/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/node-app/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/node-app/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/node-app/deployments","created_at":"2022-03-25T04:10:15Z","updated_at":"2021-12-30T22:39:01Z","pushed_at":"2022-03-11T17:49:16Z","git_url":"git://github.com/hu-weihua/node-app.git","ssh_url":"git@github.com:hu-weihua/node-app.git","clone_url":"https://github.com/hu-weihua/node-app.git","svn_url":"https://github.com/hu-weihua/node-app","homepage":null,"size":82,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":140396693,"node_id":"MDEwOlJlcG9zaXRvcnkxNDAzOTY2OTM=","name":"personal_repo","full_name":"hu-weihua/personal_repo","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/personal_repo","description":"This is repository of hwh","fork":false,"url":"https://api.github.com/repos/hu-weihua/personal_repo","forks_url":"https://api.github.com/repos/hu-weihua/personal_repo/forks","keys_url":"https://api.github.com/repos/hu-weihua/personal_repo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/personal_repo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/personal_repo/teams","hooks_url":"https://api.github.com/repos/hu-weihua/personal_repo/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/personal_repo/events","assignees_url":"https://api.github.com/repos/hu-weihua/personal_repo/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/personal_repo/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/personal_repo/tags","blobs_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/personal_repo/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/personal_repo/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/personal_repo/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/personal_repo/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/personal_repo/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/personal_repo/subscription","commits_url":"https://api.github.com/repos/hu-weihua/personal_repo/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/personal_repo/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/personal_repo/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/personal_repo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/personal_repo/merges","archive_url":"https://api.github.com/repos/hu-weihua/personal_repo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/personal_repo/downloads","issues_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/personal_repo/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/personal_repo/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/personal_repo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/personal_repo/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/personal_repo/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/personal_repo/deployments","created_at":"2018-07-10T07:49:54Z","updated_at":"2021-11-04T03:05:14Z","pushed_at":"2021-11-04T03:05:11Z","git_url":"git://github.com/hu-weihua/personal_repo.git","ssh_url":"git@github.com:hu-weihua/personal_repo.git","clone_url":"https://github.com/hu-weihua/personal_repo.git","svn_url":"https://github.com/hu-weihua/personal_repo","homepage":null,"size":45089,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":411522280,"node_id":"R_kgDOGIdU6A","name":"quay_tools","full_name":"hu-weihua/quay_tools","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_tools","description":"scripts for quay ","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_tools","forks_url":"https://api.github.com/repos/hu-weihua/quay_tools/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_tools/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_tools/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_tools/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_tools/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_tools/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_tools/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_tools/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_tools/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_tools/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_tools/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_tools/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_tools/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_tools/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_tools/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_tools/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_tools/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_tools/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_tools/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_tools/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_tools/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_tools/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_tools/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_tools/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_tools/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_tools/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_tools/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_tools/deployments","created_at":"2021-09-29T03:52:46Z","updated_at":"2021-10-14T04:53:01Z","pushed_at":"2021-10-14T04:52:58Z","git_url":"git://github.com/hu-weihua/quay_tools.git","ssh_url":"git@github.com:hu-weihua/quay_tools.git","clone_url":"https://github.com/hu-weihua/quay_tools.git","svn_url":"https://github.com/hu-weihua/quay_tools","homepage":null,"size":5,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":409051481,"node_id":"R_kgDOGGGhWQ","name":"quay_fedaro_laptop_setup","full_name":"hu-weihua/quay_fedaro_laptop_setup","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup","forks_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/deployments","created_at":"2021-09-22T03:25:19Z","updated_at":"2021-09-22T03:32:29Z","pushed_at":"2021-09-22T03:32:26Z","git_url":"git://github.com/hu-weihua/quay_fedaro_laptop_setup.git","ssh_url":"git@github.com:hu-weihua/quay_fedaro_laptop_setup.git","clone_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup.git","svn_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup","homepage":null,"size":2,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":251479333,"node_id":"MDEwOlJlcG9zaXRvcnkyNTE0NzkzMzM=","name":"cam_tools","full_name":"hu-weihua/cam_tools","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/cam_tools","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/cam_tools","forks_url":"https://api.github.com/repos/hu-weihua/cam_tools/forks","keys_url":"https://api.github.com/repos/hu-weihua/cam_tools/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/cam_tools/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/cam_tools/teams","hooks_url":"https://api.github.com/repos/hu-weihua/cam_tools/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/cam_tools/events","assignees_url":"https://api.github.com/repos/hu-weihua/cam_tools/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/cam_tools/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/cam_tools/tags","blobs_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/cam_tools/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/cam_tools/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/cam_tools/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/cam_tools/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/cam_tools/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/cam_tools/subscription","commits_url":"https://api.github.com/repos/hu-weihua/cam_tools/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/cam_tools/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/cam_tools/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/cam_tools/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/cam_tools/merges","archive_url":"https://api.github.com/repos/hu-weihua/cam_tools/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/cam_tools/downloads","issues_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/cam_tools/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/cam_tools/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/cam_tools/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/cam_tools/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/cam_tools/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/cam_tools/deployments","created_at":"2020-03-31T02:22:26Z","updated_at":"2021-07-12T10:24:29Z","pushed_at":"2021-07-12T10:24:26Z","git_url":"git://github.com/hu-weihua/cam_tools.git","ssh_url":"git@github.com:hu-weihua/cam_tools.git","clone_url":"https://github.com/hu-weihua/cam_tools.git","svn_url":"https://github.com/hu-weihua/cam_tools","homepage":null,"size":151,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":265992712,"node_id":"MDEwOlJlcG9zaXRvcnkyNjU5OTI3MTI=","name":"englist_study","full_name":"hu-weihua/englist_study","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/englist_study","description":"this is repo for english study","fork":false,"url":"https://api.github.com/repos/hu-weihua/englist_study","forks_url":"https://api.github.com/repos/hu-weihua/englist_study/forks","keys_url":"https://api.github.com/repos/hu-weihua/englist_study/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/englist_study/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/englist_study/teams","hooks_url":"https://api.github.com/repos/hu-weihua/englist_study/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/englist_study/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/englist_study/events","assignees_url":"https://api.github.com/repos/hu-weihua/englist_study/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/englist_study/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/englist_study/tags","blobs_url":"https://api.github.com/repos/hu-weihua/englist_study/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/englist_study/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/englist_study/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/englist_study/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/englist_study/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/englist_study/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/englist_study/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/englist_study/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/englist_study/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/englist_study/subscription","commits_url":"https://api.github.com/repos/hu-weihua/englist_study/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/englist_study/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/englist_study/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/englist_study/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/englist_study/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/englist_study/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/englist_study/merges","archive_url":"https://api.github.com/repos/hu-weihua/englist_study/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/englist_study/downloads","issues_url":"https://api.github.com/repos/hu-weihua/englist_study/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/englist_study/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/englist_study/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/englist_study/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/englist_study/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/englist_study/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/englist_study/deployments","created_at":"2020-05-22T01:45:08Z","updated_at":"2021-05-01T00:58:19Z","pushed_at":"2021-05-01T00:58:17Z","git_url":"git://github.com/hu-weihua/englist_study.git","ssh_url":"git@github.com:hu-weihua/englist_study.git","clone_url":"https://github.com/hu-weihua/englist_study.git","svn_url":"https://github.com/hu-weihua/englist_study","homepage":null,"size":82,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":308238023,"node_id":"MDEwOlJlcG9zaXRvcnkzMDgyMzgwMjM=","name":"rh-do380-ge","full_name":"hu-weihua/rh-do380-ge","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/rh-do380-ge","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/rh-do380-ge","forks_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/forks","keys_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/teams","hooks_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/events","assignees_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/tags","blobs_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/subscription","commits_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/merges","archive_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/downloads","issues_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/deployments","created_at":"2020-10-29T06:30:04Z","updated_at":"2020-11-06T07:14:37Z","pushed_at":"2020-11-06T07:14:35Z","git_url":"git://github.com/hu-weihua/rh-do380-ge.git","ssh_url":"git@github.com:hu-weihua/rh-do380-ge.git","clone_url":"https://github.com/hu-weihua/rh-do380-ge.git","svn_url":"https://github.com/hu-weihua/rh-do380-ge","homepage":null,"size":8389,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":309275165,"node_id":"MDEwOlJlcG9zaXRvcnkzMDkyNzUxNjU=","name":"baisc_test_data","full_name":"hu-weihua/baisc_test_data","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/baisc_test_data","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/baisc_test_data","forks_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/forks","keys_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/teams","hooks_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/events","assignees_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/tags","blobs_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/subscription","commits_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/merges","archive_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/downloads","issues_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/deployments","created_at":"2020-11-02T06:12:55Z","updated_at":"2020-11-02T07:45:40Z","pushed_at":"2020-11-02T07:45:38Z","git_url":"git://github.com/hu-weihua/baisc_test_data.git","ssh_url":"git@github.com:hu-weihua/baisc_test_data.git","clone_url":"https://github.com/hu-weihua/baisc_test_data.git","svn_url":"https://github.com/hu-weihua/baisc_test_data","homepage":null,"size":7,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":308519553,"node_id":"MDEwOlJlcG9zaXRvcnkzMDg1MTk1NTM=","name":"gitops-gitops","full_name":"hu-weihua/gitops-gitops","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/gitops-gitops","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/gitops-gitops","forks_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/forks","keys_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/teams","hooks_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/events","assignees_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/tags","blobs_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/subscription","commits_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/merges","archive_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/downloads","issues_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/deployments","created_at":"2020-10-30T03:53:46Z","updated_at":"2020-10-30T04:27:19Z","pushed_at":"2020-10-30T04:27:16Z","git_url":"git://github.com/hu-weihua/gitops-gitops.git","ssh_url":"git@github.com:hu-weihua/gitops-gitops.git","clone_url":"https://github.com/hu-weihua/gitops-gitops.git","svn_url":"https://github.com/hu-weihua/gitops-gitops","homepage":null,"size":4,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":279471706,"node_id":"MDEwOlJlcG9zaXRvcnkyNzk0NzE3MDY=","name":"DO180-apps","full_name":"hu-weihua/DO180-apps","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/DO180-apps","description":"DO180 Repository for Sample Applications","fork":true,"url":"https://api.github.com/repos/hu-weihua/DO180-apps","forks_url":"https://api.github.com/repos/hu-weihua/DO180-apps/forks","keys_url":"https://api.github.com/repos/hu-weihua/DO180-apps/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/DO180-apps/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/DO180-apps/teams","hooks_url":"https://api.github.com/repos/hu-weihua/DO180-apps/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/DO180-apps/events","assignees_url":"https://api.github.com/repos/hu-weihua/DO180-apps/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/DO180-apps/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/DO180-apps/tags","blobs_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/DO180-apps/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/DO180-apps/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/DO180-apps/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/DO180-apps/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/DO180-apps/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/DO180-apps/subscription","commits_url":"https://api.github.com/repos/hu-weihua/DO180-apps/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/DO180-apps/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/DO180-apps/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/DO180-apps/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/DO180-apps/merges","archive_url":"https://api.github.com/repos/hu-weihua/DO180-apps/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/DO180-apps/downloads","issues_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/DO180-apps/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/DO180-apps/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/DO180-apps/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/DO180-apps/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/DO180-apps/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/DO180-apps/deployments","created_at":"2020-07-14T03:29:50Z","updated_at":"2020-07-14T03:29:51Z","pushed_at":"2020-07-17T08:41:06Z","git_url":"git://github.com/hu-weihua/DO180-apps.git","ssh_url":"git@github.com:hu-weihua/DO180-apps.git","clone_url":"https://github.com/hu-weihua/DO180-apps.git","svn_url":"https://github.com/hu-weihua/DO180-apps","homepage":null,"size":230,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":238137702,"node_id":"MDEwOlJlcG9zaXRvcnkyMzgxMzc3MDI=","name":"mig-e2e","full_name":"hu-weihua/mig-e2e","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/mig-e2e","description":"End to end tests for OCP Migration","fork":true,"url":"https://api gunicorn-web stdout | .github.com/repos/hu-weihua/mig-e2e","forks_url":"https://api.github.com/repos/hu-weihua/mig-e2e/forks","keys_url":"https://api.github.com/repos/hu-weihua/mig-e2e/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/mig-e2e/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/mig-e2e/teams","hooks_url":"https://api.github.com/repos/hu-weihua/mig-e2e/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/mig-e2e/events","assignees_url":"https://api.github.com/repos/hu-weihua/mig-e2e/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/mig-e2e/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/mig-e2e/tags","blobs_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/mig-e2e/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/mig-e2e/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/mig-e2e/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/mig-e2e/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/mig-e2e/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/mig-e2e/subscription","commits_url":"https://api.github.com/repos/hu-weihua/mig-e2e/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/mig-e2e/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/mig-e2e/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/mig-e2e/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/mig-e2e/merges","archive_url":"https://api.github.com/repos/hu-weihua/mig-e2e/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/mig-e2e/downloads","issues_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/mig-e2e/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/mig-e2e/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/mig-e2e/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/mig-e2e/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/mig-e2e/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/mig-e2e/deployments","created_at":"2020-02-04T06:21:19Z","updated_at":"2020-02-25T17:18:34Z","pushed_at":"2020-04-28T01:45:39Z","git_url":"git://github.com/hu-weihua/mig-e2e.git","ssh_url":"git@github.com:hu-weihua/mig-e2e.git","clone_url":"https://github.com/hu-weihua/mig-e2e.git","svn_url":"https://github.com/hu-weihua/mig-e2e","homepage":null,"size":211,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":228530382,"node_id":"MDEwOlJlcG9zaXRvcnkyMjg1MzAzODI=","name":"mig-operator","full_name":"hu-weihua/mig-operator","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/mig-operator","description":"OpenShift Migration Operator","fork":true,"url":"https://api.github.com/repos/hu-weihua/mig-operator","forks_url":"https://api.github.com/repos/hu-weihua/mig-operator/forks","keys_url":"https://api.github.com/repos/hu-weihua/mig-operator/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/mig-operator/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/mig-operator/teams","hooks_url":"https://api.github.com/repos/hu-weihua/mig-operator/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/mig-operator/events","assignees_url":"https://api.github.com/repos/hu-weihua/mig-operator/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/mig-operator/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/mig-operator/tags","blobs_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/mig-operator/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/mig-operator/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/mig-operator/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/mig-operator/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/mig-operator/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/mig-operator/subscription","commits_url":"https://api.github.com/repos/hu-weihua/mig-operator/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/mig-operator/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/mig-operator/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/mig-operator/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/mig-operator/merges","archive_url":"https://api.github.com/repos/hu-weihua/mig-operator/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/mig-operator/downloads","issues_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/mig-operator/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/mig-operator/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/mig-operator/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/mig-operator/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/mig-operator/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/mig-operator/deployments","created_at":"2019-12-17T04:09:11Z","updated_at":"2020-02-25T14:54:53Z","pushed_at":"2019-12-16T22:52:14Z","git_url":"git://github.com/hu-weihua/mig-operator.git","ssh_url":"git@github.com:hu-weihua/mig-operator.git","clone_url":"https://github.com/hu-weihua/mig-operator.git","svn_url":"https://github.com/hu-weihua/mig-operator","homepage":"","size":15667,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":235289269,"node_id":"MDEwOlJlcG9zaXRvcnkyMzUyODkyNjk=","name":"cam_test","full_name":"hu-weihua/cam_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/cam_test","description":"just for cam test","fork":false,"url":"https://api.github.com/repos/hu-weihua/cam_test","forks_url":"https://api.github.com/repos/hu-weihua/cam_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/cam_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/cam_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/cam_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/cam_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/cam_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/cam_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/cam_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/cam_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/cam_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/cam_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/cam_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/cam_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/cam_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/cam_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/cam_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/cam_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/cam_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/cam_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/cam_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/cam_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/cam_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/cam_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/cam_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/cam_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/cam_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/cam_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/cam_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/cam_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/cam_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/cam_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/cam_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/cam_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/cam_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/cam_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/cam_test/deployments","created_at":"2020-01-21T08:21:49Z","updated_at":"2020-01-21T11:46:02Z","pushed_at":"2020-01-21T11:46:00Z","git_url":"git://github.com/hu-weihua/cam_test.git","ssh_url":"git@github.com:hu-weihua/cam_test.git","clone_url":"https://github.com/hu-weihua/cam_test.git","svn_url":"https://github.com/hu-weihua/cam_test","homepage":null,"size":13,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":231495393,"node_id":"MDEwOlJlcG9zaXRvcnkyMzE0OTUzOTM=","name":"Ansible_projects","full_name":"hu-weihua/Ansible_projects","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/Ansible_projects","description":"To save some ansible projects","fork":false,"url":"https://api.github.com/repos/hu-weihua/Ansible_projects","forks_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/forks","keys_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/teams","hooks_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/events","assignees_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/tags","blobs_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/subscription","commits_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/merges","archive_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/downloads","issues_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/deployments","created_at":"2020-01-03T02:20:16Z","updated_at":"2020-01-03T09:09:29Z","pushed_at":"2020-01-03T09:09:27Z","git_url":"git://github.com/hu-weihua/Ansible_projects.git","ssh_url":"git@github.com:hu-weihua/Ansible_projects.git","clone_url":"https://github.com/hu-weihua/Ansible_projects.git","svn_url":"https://github.com/hu-weihua/Ansible_projects","homepage":null,"size":19,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":88809435,"node_id":"MDEwOlJlcG9zaXRvcnk4ODgwOTQzNQ==","name":"xcat-process","full_name":"hu-weihua/xcat-process","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-process","description":"The process, guidence, rule, tips need to follow by xCAT developers and FVTs","fork":false,"url":"https://api.github.com/repos/hu-weihua/xcat-process","forks_url":"https://api.github.com/repos/hu-weihua/xcat-process/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-process/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-process/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-process/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-process/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-process/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-process/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-process/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-process/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-process/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-process/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-process/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-process/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-process/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-process/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-process/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-process/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-process/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-process/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-process/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-process/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-process/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-process/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-process/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-process/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-process/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-process/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-process/deployments","created_at":"2017-04-20T01:58:16Z","updated_at":"2019-06-25T18:55:32Z","pushed_at":"2017-04-19T09:45:48Z","git_url":"git://github.com/hu-weihua/xcat-process.git","ssh_url":"git@github.com:hu-weihua/xcat-process.git","clone_url":"https://github.com/hu-weihua/xcat-process.git","svn_url":"https://github.com/hu-weihua/xcat-process","homepage":null,"size":337,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":2,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":2,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":40430481,"node_id":"MDEwOlJlcG9zaXRvcnk0MDQzMDQ4MQ==","name":"xcat-core","full_name":"hu-weihua/xcat-core","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-core","description":"Code repo for xCAT core packages","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-core","forks_url":"https://api.github.com/repos/hu-weihua/xcat-core/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-core/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-core/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-core/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-core/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-core/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-core/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-core/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-core/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-core/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-core/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-core/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-core/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-core/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-core/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-core/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-core/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-core/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-core/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-core/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-core/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-core/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-core/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-core/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-core/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-core/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-core/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-core/deployments","created_at":"2015-08-09T08:43:37Z","updated_at":"2019-05-23T09:48:55Z","pushed_at":"2019-05-23T09:48:47Z","git_url":"git://github.com/hu-weihua/xcat-core.git","ssh_url":"git@github.com:hu-weihua/xcat-core.git","clone_url":"https://github.com/hu-weihua/xcat-core.git","svn_url":"https://github.com/hu-weihua/xcat-core","homepage":null,"size":204915,"stargazers_count":0,"watchers_count":0,"language":"Perl","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":184030584,"node_id":"MDEwOlJlcG9zaXRvcnkxODQwMzA1ODQ=","name":"xcat-docker","full_name":"hu-weihua/xcat-docker","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-docker","description":"Repository to build Docker image for xCAT","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-docker","forks_url":"https://api.github.com/repos/hu-weihua/xcat-docker/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-docker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-docker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-docker/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-docker/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-docker/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-docker/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-docker/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-docker/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-docker/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-docker/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-docker/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-docker/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-docker/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-docker/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-docker/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-docker/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-docker/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-docker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-docker/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-docker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-docker/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-docker/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-docker/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-docker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-docker/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-docker/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-docker/deployments","created_at":"2019-04-29T08:25:05Z","updated_at":"2019-05-17T10:33:30Z","pushed_at":"2019-05-23T10:38:53Z","git_url":"git://github.com/hu-weihua/xcat-docker.git","ssh_url":"git@github.com:hu-weihua/xcat-docker.git","clone_url":"https://github.com/hu-weihua/xcat-docker.git","svn_url":"https://github.com/hu-weihua/xcat-docker","homepage":"","size":22,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":178361168,"node_id":"MDEwOlJlcG9zaXRvcnkxNzgzNjExNjg=","name":"xcat2.github.io","full_name":"hu-weihua/xcat2.github.io","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat2.github.io","description":"The repo manages the xcat2 home page: https://xcat2.github.io/","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat2.github.io","forks_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/deployments","created_at":"2019-03-29T08:09:00Z","updated_at":"2019-03-29T08:09:02Z","pushed_at":"2019-03-29T08:25:09Z","git_url":"git://github.com/hu-weihua/xcat2.github.io.git","ssh_url":"git@github.com:hu-weihua/xcat2.github.io.git","clone_url":"https://github.com/hu-weihua/xcat2.github.io.git","svn_url":"https://github.com/hu-weihua/xcat2.github.io","homepage":"","size":35142,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":91755483,"node_id":"MDEwOlJlcG9zaXRvcnk5MTc1NTQ4Mw==","name":"xcat.org","full_name":"hu-weihua/xcat.org","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat.org","description":"Repository for managing the xcat.org website ","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat.org","forks_url":"https://api.github.com/repos/hu-weihua/xcat.org/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat.org/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat.org/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat.org/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat.org/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat.org/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat.org/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat.org/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat.org/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat.org/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat.org/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat.org/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat.org/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat.org/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat.org/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat.org/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat.org/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat.org/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat.org/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat.org/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat.org/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat.org/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat.org/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat.org/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat.org/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat.org/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat.org/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat.org/deployments","created_at":"2017-05-19T02:07:52Z","updated_at":"2018-09-27T06:11:32Z","pushed_at":"2018-09-27T06:11:31Z","git_url":"git://github.com/hu-weihua/xcat.org.git","ssh_url":"git@github.com:hu-weihua/xcat.org.git","clone_url":"https://github.com/hu-weihua/xcat.org.git","svn_url":"https://github.com/hu-weihua/xcat.org","homepage":null,"size":159,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":144540798,"node_id":"MDEwOlJlcG9zaXRvcnkxNDQ1NDA3OTg=","name":"travis_ci_test","full_name":"hu-weihua/travis_ci_test","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/travis_ci_test","description":"This repository is used to test travis ci","fork":false,"url":"https://api.github.com/repos/hu-weihua/travis_ci_test","forks_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/deployments","created_at":"2018-08-13T06:54:07Z","updated_at":"2018-08-17T08:53:39Z","pushed_at":"2018-08-17T08:53:38Z","git_url":"git://github.com/hu-weihua/travis_ci_test.git","ssh_url":"git@github.com:hu-weihua/travis_ci_test.git","clone_url":"https://github.com/hu-weihua/travis_ci_test.git","svn_url":"https://github.com/hu-weihua/travis_ci_test","homepage":null,"size":22,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"has_discussions":false,"forks_count":1,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":1,"open_issues":1,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":131094324,"node_id":"MDEwOlJlcG9zaXRvcnkxMzEwOTQzMjQ=","name":"goconserver","full_name":"hu-weihua/goconserver","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/goconserver","description":"Microservice based console server to help log and redirect the terminal content for multiple session hosts.","fork":true,"url":"https://api.github.com/repos/hu-weihua/goconserver","forks_url":"https://api.github.com/repos/hu-weihua/goconserver/forks","keys_url":"https://api.github.com/repos/hu-weihua/goconserver/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/goconserver/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/goconserver/teams","hooks_url":"https://api.github.com/repos/hu-weihua/goconserver/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/goconserver/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/goconserver/events","assignees_url":"https://api.github.com/repos/hu-weihua/goconserver/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/goconserver/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/goconserver/tags","blobs_url":"https://api.github.com/repos/hu-weihua/goconserver/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/goconserver/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/goconserver/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/goconserver/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/goconserver/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/goconserver/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/goconserver/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/goconserver/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/goconserver/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/goconserver/subscription","commits_url":"https://api.github.com/repos/hu-weihua/goconserver/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/goconserver/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/goconserver/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/goconserver/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/goconserver/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/goconserver/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/goconserver/merges","archive_url":"https://api.github.com/repos/hu-weihua/goconserver/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/goconserver/downloads","issues_url":"https://api.github.com/repos/hu-weihua/goconserver/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/goconserver/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/goconserver/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/goconserver/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/goconserver/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/goconserver/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/goconserver/deployments","created_at":"2018-04-26T03:21:26Z","updated_at":"2018-04-26T03:21:28Z","pushed_at":"2018-04-17T04:11:25Z","git_url":"git://github.com/hu-weihua/goconserver.git","ssh_url":"git@github.com:hu-weihua/goconserver.git","clone_url":"https://github.com/hu-weihua/goconserver.git","svn_url":"https://github.com/hu-weihua/goconserver","homepage":"","size":6913,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":131093726,"node_id":"MDEwOlJlcG9zaXRvcnkxMzEwOTM3MjY=","name":"xcat-inventory","full_name":"hu-weihua/xcat-inventory","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-inventory","description":"an inventory tool for xcat cluster","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-inventory","forks_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/deployments","created_at":"2018-04-26T03:15:17Z","updated_at":"2018-04-26T03:15:19Z","pushed_at":"2018-04-24T20:30:44Z","git_url":"git://github.com/hu-weihua/xcat-inventory.git","ssh_url":"git@github.com:hu-weihua/xcat-inventory.git","clone_url":"https://github.com/hu-weihua/xcat-inventory.git","svn_url":"https://github.com/hu-weihua/xcat-inventory","homepage":null,"size":357,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":40395956,"node_id":"MDEwOlJlcG9zaXRvcnk0MDM5NTk1Ng==","name":"xcat-doc","full_name":"hu-weihua/xcat-doc","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-doc","description":"xCAT Documentation","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-doc","forks_url":"https://api.github.com/repos/hu-weihua/xcat-doc/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-doc/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-doc/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-doc/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-doc/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-doc/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-doc/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-doc/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-doc/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-doc/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-doc/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-doc/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-doc/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-doc/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-doc/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-doc/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-doc/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-doc/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-doc/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-doc/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-doc/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-doc/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-doc/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-doc/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-doc/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-doc/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-doc/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-doc/deployments","created_at":"2015-08-08T07:59:40Z","updated_at":"2015-07-23T02:45:40Z","pushed_at":"2015-08-10T07:06:39Z","git_url":"git://github.com/hu-weihua/xcat-doc.git","ssh_url":"git@github.com:hu-weihua/xcat-doc.git","clone_url":"https://github.com/hu-weihua/xcat-doc.git","svn_url":"https://githu gunicorn-web stdout | b.com/hu-weihua/xcat-doc","homepage":null,"size":4276,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}}] gunicorn-web stdout | 2023-08-08 05:51:02,035 [199] [DEBUG] [app] Ending request: urn:request:f9b5929b-a2d5-4949-9fa5-d20cf789fb85 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/sources) {'endpoint': 'api.buildtriggersources', 'request_id': 'urn:request:f9b5929b-a2d5-4949-9fa5-d20cf789fb85', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/sources', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/sources', 'parameters': {}, 'json_body': {'namespace': 'hu-weihua'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:51:02,036 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:02,036 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:51:02 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/sources HTTP/1.0" 200 6085 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:51:02 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/sources HTTP/2.0" 200 6085 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.862 566 0.863) notificationworker stdout | 2023-08-08 05:51:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:51:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:05.371682+00:00 (in 3.192624 seconds) notificationworker stdout | 2023-08-08 05:51:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:02 UTC)" (scheduled at 2023-08-08 05:51:02.178530+00:00) notificationworker stdout | 2023-08-08 05:51:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:51:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:51:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:51:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:12.293757+00:00 (in 9.999421 seconds) chunkcleanupworker stdout | 2023-08-08 05:51:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:12 UTC)" (scheduled at 2023-08-08 05:51:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:51:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:51:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 2, 294751), True, datetime.datetime(2023, 8, 8, 5, 51, 2, 294751), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:51:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:51:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:51:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:51:02,876 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:51:02,885 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:51:02,887 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:51:02,890 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:51:02,894 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:51:02,897 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:51:02,901 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:51:02,904 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:51:02,907 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-web stdout | 2023-08-08 05:51:04,047 [197] [DEBUG] [app] Starting request: urn:request:dcd30561-9f7f-437c-9995-8dce7b49daac (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/fields/refs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:51:04,047 [199] [DEBUG] [app] Starting request: urn:request:17f9ee22-7428-4d5a-b8af-53a7dd408f53 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/subdir) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:51:04,047 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,047 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,047 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,047 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,048 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,048 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,055 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:51:04,055 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:51:04,055 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:51:04,055 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:51:04,056 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,056 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,056 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:51:04,056 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:51:04,056 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,056 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,056 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:04,056 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:04,056 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:04,056 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:04,056 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:04,056 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:04,058 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:51:04,058 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:51:04,065 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:51:04,065 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:51:04,066 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,066 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,069 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,070 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,072 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,073 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,075 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,075 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:04,076 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:04,077 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:51:04,078 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 repositorygcworker stdout | 2023-08-08 05:51:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:51:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:09.793658+00:00 (in 5.605670 seconds) repositorygcworker stdout | 2023-08-08 05:51:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:04 UTC)" (scheduled at 2023-08-08 05:51:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:51:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:51:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:04 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:51:04,284 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:04,285 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4995', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '5', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'B0A5:4043:DC12C0:1C3D72A:64D1D7C8'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAVIJZA4O4ZD2VEAXEDE2HMPI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:51:04,289 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:51:04,290 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:04,290 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4994', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '6', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'F1C3:9C0A:D860A0:1BC96F8:64D1D7C8'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAVIJZA4O4ZD2VEAXEDE2HMPI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:51:04,381 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:04,382 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"8f50b6eed3d311f807e06b202890c0715d888c822ddb9cf66d1d8dbf3ac4d22e"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4993', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '7', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'F1C3:9C0A:D860EB:1BC978E:64D1D7C8'} [{"name":"branch1","commit":{"sha":"1d34effde035584623f29666b8f41b42e020ad4a","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/1d34effde035584623f29666b8f41b42e020ad4a"},"protected":false},{"name":"branch2","commit":{"sha":"c6ef6983d9344c211af3fc7312bddccbe7c0dd7c","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/c6ef6983d9344c211af3fc7312bddccbe7c0dd7c"},"protected":false},{"name":"hu-weihua-patch-1","commit":{"sha":"7f19019c0b0bf7eb48b55479771b213e8081c884","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/7f19019c0b0bf7eb48b55479771b213e8081c884"},"protected":false},{"name":"hu-weihua-patch-2","commit":{"sha":"65e01fcd85dfe53a2d52b5f7e188fca3f30daf78","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/65e01fcd85dfe53a2d52b5f7e188fca3f30daf78"},"protected":false},{"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d"},"protected":false}] gunicorn-web stdout | 2023-08-08 05:51:04,386 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:51:04,432 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:04,433 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4992', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '8', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9478:3795:DE34DA:1C8645E:64D1D7C8'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAVIJZA4O4ZD2VEAXEDE2HMPI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:51:04,512 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:04,513 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"8f50b6eed3d311f807e06b202890c0715d888c822ddb9cf66d1d8dbf3ac4d22e"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4991', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '9', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9478:3795:DE3504:1C86491:64D1D7C8'} [{"name":"branch1","commit":{"sha":"1d34effde035584623f29666b8f41b42e020ad4a","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/1d34effde035584623f29666b8f41b42e020ad4a"},"protected":false},{"name":"branch2","commit":{"sha":"c6ef6983d9344c211af3fc7312bddccbe7c0dd7c","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/c6ef6983d9344c211af3fc7312bddccbe7c0dd7c"},"protected":false},{"name":"hu-weihua-patch-1","commit":{"sha":"7f19019c0b0bf7eb48b55479771b213e8081c884","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/7f19019c0b0bf7eb48b55479771b213e8081c884"},"protected":false},{"name":"hu-weihua-patch-2","commit":{"sha":"65e01fcd85dfe53a2d52b5f7e188fca3f30daf78","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/65e01fcd85dfe53a2d52b5f7e188fca3f30daf78"},"protected":false},{"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d"},"protected":false}] gunicorn-web stdout | 2023-08-08 05:51:04,515 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:04,516 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4990', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '10', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9480:7865:DAE4C7:1C1A605:64D1D7C8'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAVIJZA4O4ZD2VEAXEDE2HMPI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:51:04,598 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/tags HTTP/1.1" 200 2 gunicorn-web stdout | 2023-08-08 05:51:04,599 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/tags {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '2', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"58a2bb6ecc8fce6df642c54032b9e7e4046a3bfca31936aaa7d3174e97029657"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4989', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '11', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': '9480:7865:DAE4E8:1C1A64D:64D1D7C8'} [] gunicorn-web stdout | 2023-08-08 05:51:04,602 [197] [DEBUG] [app] Ending request: urn:request:dcd30561-9f7f-437c-9995-8dce7b49daac (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/fields/refs) {'endpoint': 'api.buildtriggerfieldvalues', 'request_id': 'urn:request:dcd30561-9f7f-437c-9995-8dce7b49daac', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/fields/refs', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/fields/refs', 'parameters': {}, 'json_body': {'build_source': 'hu-weihua/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:51:04,602 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:04,603 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:51:04 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/fields/refs HTTP/1.0" 200 225 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:51:04 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/fields/refs HTTP/2.0" 200 225 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.558 497 0.558) gunicorn-web stdout | 2023-08-08 05:51:04,645 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches/main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:04,646 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches/main {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"b19a96da3bd4ede82608d2b31f673c1dd21011da1ecf97ad3c7fbcc0144bdb98"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4988', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '12', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'B0A5:4043:DC135E:1C3D84D:64D1D7C8'} {"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","node_id":"C_kwDOG6AIe9oAKDg4MGZiNzAwZjhiNTEwM2U3YWRjMjBmMDgwMzg0MzRlOWY3MTJjNWQ","commit":{"author":{"name":"Hu Weihua","email":"hu_weihua@163.com","date":"2023-08-08T02:50:20Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2023-08-08T02:50:20Z"},"message":"Update update_commit.sh","tree":{"sha":"c5abbb94e6edfb389f732ea578c71d6db692406d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/c5abbb94e6edfb389f732ea578c71d6db692406d"},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsBcBAABCAAQBQJk0a1sCRBK7hj4Ov3rIwAA5MMIABciqM6Y7qngb0Joeh7FSIAp\nnv0pckL5M+0g6IBcVfxG4RC6Hfti5Gs1lYKBppBJMF5OUVdITpe2GOii6WcGieL9\nbo/Acx0X4liVp4QzoQQTNYRmCZFODrlb/FVuVcdHRtBO/hXNAsqmPOHGC1Guf5lG\nHWpLPZF9e8lnbI1/MIiu4O5+OK+qRw26pEbRHjukBa921OQ/27s+MATflSKJ+bBJ\nBshN6stBgyAA4kDsPwl7iIHn1xSGhADLw3q8BnDRIT5jRHzDoqHdck9VpVatNxxL\nK+9HZiaHhyQ3+CsvT5/NNMy5cWoqJvSpjNobgmw4bYg70AB2xUHy85jPOl7Rizk=\n=DSm6\n-----END PGP SIGNATURE-----\n","payload":"tree c5abbb94e6edfb389f732ea578c71d6db692406d\nparent f0030ac4a54f1c210daf84729cfd233407d43d14\nauthor Hu Weihua 1691463020 +0800\ncommitter GitHub 1691463020 +0800\n\nUpdate update_commit.sh"}},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","html_url":"https://github.com/hu-weihua/quay_build_test/commit/880fb700f8b5103e7adc20f08038434e9f712c5d","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d/comments","author":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","site_admin":false},"parents":[{"sha":"f0030ac4a54f1c210daf84729cfd233407d43d14","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/f0030ac4a54f1c210daf84729cfd233407d43d14","html_url":"https://github.com/hu-weihua/quay_build_test/commit/f0030ac4a54f1c210daf84729cfd233407d43d14"}]},"_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/branches/main","html":"https://github.com/hu-weihua/quay_build_test/tree/main"},"protected":false,"protection":{"enabled":false,"required_status_checks":{"enforcement_level":"off","contexts":[],"checks":[]}},"protection_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches/main/protection"} gunicorn-web stdout | 2023-08-08 05:51:04,740 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d?recursive=1 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:04,741 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d?recursive=1 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:04 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=86400, s-maxage=86400', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"e4cede27df417473fd2d3da2a0bb0f64d9c4f23c5a192ecaf6dd873fd2c8f62c"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4987', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '13', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'B0A5:4043:DC1399:1C3D8BC:64D1D7C8'} {"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d","tree":[{"path":"Dockerfile","mode":"100644","type":"blob","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6"},{"path":"README.md","mode":"100644","type":"blob","sha":"929a3f9add2efe6f5d73c5226dc603686c63a202","size":57,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/929a3f9add2efe6f5d73c5226dc603686c63a202"},{"path":"aaa","mode":"100644","type":"blob","sha":"73cf2b7c923ab8e99ff19403d3b7c4191b2460a3","size":20,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/73cf2b7c923ab8e99ff19403d3b7c4191b2460a3"},{"path":"newfile5540167606144473","mode":"100644","type":"blob","sha":"aa5d92554efab5c3075f098e51bd8dc9642d6075","size":22,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/aa5d92554efab5c3075f098e51bd8dc9642d6075"},{"path":"newfile7189991658911539","mode":"100644","type":"blob","sha":"894bdc5476f8f12f6e90c706cc91aa84cf327231","size":22,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/894bdc5476f8f12f6e90c706cc91aa84cf327231"},{"path":"update_commit.sh","mode":"100644","type":"blob","sha":"8ac65cd6da3c09ee9b225fddc5eea5023f65c6aa","size":69,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/8ac65cd6da3c09ee9b225fddc5eea5023f65c6aa"}],"truncated":false} gunicorn-web stdout | 2023-08-08 05:51:04,744 [199] [DEBUG] [app] Ending request: urn:request:17f9ee22-7428-4d5a-b8af-53a7dd408f53 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/subdir) {'endpoint': 'api.buildtriggersubdirs', 'request_id': 'urn:request:17f9ee22-7428-4d5a-b8af-53a7dd408f53', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/subdir', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/subdir', 'parameters': {}, 'json_body': {'build_source': 'hu-weihua/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:51:04,745 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:51:04 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/subdir HTTP/2.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.701 288 0.701) gunicorn-web stdout | 2023-08-08 05:51:04,746 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:51:04 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/subdir HTTP/1.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 05:51:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:51:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:15.371682+00:00 (in 9.999419 seconds) notificationworker stdout | 2023-08-08 05:51:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:15 UTC)" (scheduled at 2023-08-08 05:51:05.371682+00:00) notificationworker stdout | 2023-08-08 05:51:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:51:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 5, 373007), True, datetime.datetime(2023, 8, 8, 5, 51, 5, 373007), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:51:05,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:51:05,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:51:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:51:07,179 [200] [DEBUG] [app] Starting request: urn:request:162d7b82-2f99-436d-b0a6-111418434c3f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:51:07,180 [198] [DEBUG] [app] Starting request: urn:request:b7c27eff-d102-41ed-af80-fab4146d2ce5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:51:07,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:07,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:07,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:51:07,185 [203] [DEBUG] [app] Starting request: urn:request:03cdde2b-9ee8-43fc-ab41-481158f94aac (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:51:07,186 [203] [DEBUG] [app] Ending request: urn:request:03cdde2b-9ee8-43fc-ab41-481158f94aac (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:03cdde2b-9ee8-43fc-ab41-481158f94aac', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:51:07,186 [207] [DEBUG] [app] Starting request: urn:request:d3265ac2-2484-493f-99d1-8a1b4baa9a50 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:51:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:51:07,187 [207] [DEBUG] [app] Ending request: urn:request:d3265ac2-2484-493f-99d1-8a1b4baa9a50 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d3265ac2-2484-493f-99d1-8a1b4baa9a50', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:51:07,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:51:07,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:07,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:07,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:07,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:07,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:07,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:07,192 [198] [DEBUG] [app] Starting request: urn:request:01ff6ccd-cf6f-4550-96d9-6ab9c0d66473 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:51:07,193 [198] [DEBUG] [app] Ending request: urn:request:01ff6ccd-cf6f-4550-96d9-6ab9c0d66473 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:01ff6ccd-cf6f-4550-96d9-6ab9c0d66473', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:51:07,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:07,193 [199] [DEBUG] [app] Starting request: urn:request:48c50d85-c406-4e94-8e85-2d73366dc627 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:51:07,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:07,193 [199] [DEBUG] [app] Ending request: urn:request:48c50d85-c406-4e94-8e85-2d73366dc627 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:48c50d85-c406-4e94-8e85-2d73366dc627', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:51:07,194 [198] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:51:07,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:07,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:51:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:51:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:37.197412+00:00 (in 29.999493 seconds) buildlogsarchiver stdout | 2023-08-08 05:51:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:37 UTC)" (scheduled at 2023-08-08 05:51:07.197412+00:00) gunicorn-web stdout | 2023-08-08 05:51:07,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:51:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:51:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 51, 7, 198827), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:07,204 [198] [DEBUG] [app] Ending request: urn:request:b7c27eff-d102-41ed-af80-fab4146d2ce5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b7c27eff-d102-41ed-af80-fab4146d2ce5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:51:07,204 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:07,205 [200] [DEBUG] [app] Ending request: urn:request:162d7b82-2f99-436d-b0a6-111418434c3f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:162d7b82-2f99-436d-b0a6-111418434c3f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:51:07,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:07,205 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:51:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:51:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:51:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:51:07,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:51:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:51:07,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:51:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:51:07,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:37 UTC)" executed successfully gcworker stdout | 2023-08-08 05:51:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:51:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:19.417660+00:00 (in 9.999507 seconds) gcworker stdout | 2023-08-08 05:51:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:19 UTC)" (scheduled at 2023-08-08 05:51:09.417660+00:00) gcworker stdout | 2023-08-08 05:51:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264269418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:51:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:51:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:51:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:51:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:51:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:04.187429+00:00 (in 54.393226 seconds) repositorygcworker stdout | 2023-08-08 05:51:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:09 UTC)" (scheduled at 2023-08-08 05:51:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:51:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:51:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 9, 794921), True, datetime.datetime(2023, 8, 8, 5, 51, 9, 794921), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:51:09,804 [75] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2023, 8, 8, 8, 51, 9, 794921), 1, '4959aa60-0868-424d-ba7a-77f93190b36b', 1, 'bb4cc934-5b95-44f5-845c-0b3038d59ca6']) repositorygcworker stdout | 2023-08-08 05:51:09,829 [75] [DEBUG] [workers.queueworker] Queue gave us some work: {"marker_id": 1, "original_name": "gitlab"} repositorygcworker stdout | 2023-08-08 05:51:09,830 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:51:09,830 [75] [DEBUG] [util.locking] Acquiring global lock LARGE_GARBAGE_COLLECTION repositorygcworker stdout | 2023-08-08 05:51:09,830 [75] [DEBUG] [redis_lock.acquire] Getting 'lock:LARGE_GARBAGE_COLLECTION' ... repositorygcworker stdout | 2023-08-08 05:51:09,831 [75] [INFO] [redis_lock.acquire] Got lock for 'lock:LARGE_GARBAGE_COLLECTION'. repositorygcworker stdout | 2023-08-08 05:51:09,831 [75] [DEBUG] [util.locking] Acquired lock LARGE_GARBAGE_COLLECTION repositorygcworker stdout | 2023-08-08 05:51:09,831 [75] [DEBUG] [__main__] Got repository GC queue item: {'marker_id': 1, 'original_name': 'gitlab'} repositorygcworker stdout | 2023-08-08 05:51:09,831 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."marked", "t1"."original_name", "t1"."queue_id" FROM "deletedrepository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) repositorygcworker stdout | 2023-08-08 05:51:09,837 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) repositorygcworker stdout | 2023-08-08 05:51:09,839 [75] [DEBUG] [__main__] Purging repository 1 repositorygcworker stdout | 2023-08-08 05:51:09,839 [75] [DEBUG] [peewee] ('UPDATE "repository" SET "namespace_user_id" = %s, "name" = %s, "visibility_id" = %s, "description" = %s, "badge_token" = %s, "kind_id" = %s, "trust_enabled" = %s, "state" = %s WHERE ("repository"."id" = %s)', [2, 'ec080677-2434-4b6f-b264-cdeda28a0087', 2, '', '0fe857a5-447d-43a8-9f00-b707338bb0d3', 1, False, 3, 1]) repositorygcworker stdout | 2023-08-08 05:51:09,842 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) repositorygcworker stdout | 2023-08-08 05:51:09,843 [75] [DEBUG] [data.model.gc] Purging repository 1 repositorygcworker stdout | 2023-08-08 05:51:09,843 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id" FROM "tag" AS "t1" WHERE ("t1"."repository_id" = %s) LIMIT %s', [1, 10]) repositorygcworker stdout | 2023-08-08 05:51:09,844 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."blob_id", "t1"."uploaded_at", "t1"."expires_at" FROM "uploadedblob" AS "t1" WHERE ("t1"."repository_id" = %s) LIMIT %s', [1, 10]) repositorygcworker stdout | 2023-08-08 05:51:09,846 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" WHERE ("t1"."repository_id" = %s) LIMIT %s', [1, 10]) repositorygcworker stdout | 2023-08-08 05:51:09,847 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "tag" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,848 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "repositorytag" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,849 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "manifest" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,850 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "manifestblob" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,851 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "uploadedblob" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,853 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "image" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,854 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "apprtag" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,855 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "tag" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,857 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "repositorytag" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,858 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "manifest" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,859 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "manifestblob" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,860 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "uploadedblob" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,860 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "manifestsecuritystatus" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,862 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "image" AS "t1" WHERE ("t1"."repository_id" = %s)) AS "_wrapped"', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,862 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repositorypermission" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,864 [75] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "repositorypermission" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."id" <= %s))', [1, 501]) repositorygcworker stdout | 2023-08-08 05:51:09,865 [75] [DEBUG] [peewee] ('DELETE FROM "repositorypermission" WHERE ((("repositorypermission"."repository_id" = %s) AND ("repositorypermission"."id" >= %s)) AND ("repositorypermission"."id" <= %s))', [1, 1, 1]) repositorygcworker stdout | 2023-08-08 05:51:09,867 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repositorypermission" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,869 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repositorybuild" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,870 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repositorybuildtrigger" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,871 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repositoryactioncount" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,872 [75] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."id" <= %s))', [1, 501]) repositorygcworker stdout | 2023-08-08 05:51:09,873 [75] [DEBUG] [peewee] ('DELETE FROM "repositoryactioncount" WHERE ((("repositoryactioncount"."repository_id" = %s) AND ("repositoryactioncount"."id" >= %s)) AND ("repositoryactioncount"."id" <= %s))', [1, 1, 1]) repositorygcworker stdout | 2023-08-08 05:51:09,876 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repositoryactioncount" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,877 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "star" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,878 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "accesstoken" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,879 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repositorynotification" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,880 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "blobupload" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,881 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repomirrorconfig" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,882 [75] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "repositoryauthorizedemail" AS "t1" WHERE ("t1"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,883 [75] [DEBUG] [peewee] ('DELETE FROM "deletedrepository" WHERE ("deletedrepository"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,886 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) repositorygcworker stdout | 2023-08-08 05:51:09,888 [75] [DEBUG] [peewee] ('DELETE FROM "tagmanifestlabel" WHERE ("tagmanifestlabel"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,889 [75] [DEBUG] [peewee] ('DELETE FROM "tagtorepositorytag" WHERE ("tagtorepositorytag"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,890 [75] [DEBUG] [peewee] ('DELETE FROM "repositorybuild" WHERE ("repositorybuild"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,891 [75] [DEBUG] [peewee] ('DELETE FROM "tag" WHERE ("tag"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,892 [75] [DEBUG] [peewee] ('DELETE FROM "repositorytag" WHERE ("repositorytag"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,892 [75] [DEBUG] [peewee] ('DELETE FROM "repositorybuildtrigger" WHERE ("repositorybuildtrigger"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,893 [75] [DEBUG] [peewee] ('DELETE FROM "repomirrorconfig" WHERE ("repomirrorconfig"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,894 [75] [DEBUG] [peewee] ('DELETE FROM "manifestsecuritystatus" WHERE ("manifestsecuritystatus"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,894 [75] [DEBUG] [peewee] ('DELETE FROM "manifestlegacyimage" WHERE ("manifestlegacyimage"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,895 [75] [DEBUG] [peewee] ('DELETE FROM "manifestlabel" WHERE ("manifestlabel"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,896 [75] [DEBUG] [peewee] ('DELETE FROM "manifestchild" WHERE ("manifestchild"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,897 [75] [DEBUG] [peewee] ('DELETE FROM "manifestblob" WHERE ("manifestblob"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,898 [75] [DEBUG] [peewee] ('DELETE FROM "uploadedblob" WHERE ("uploadedblob"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,899 [75] [DEBUG] [peewee] ('DELETE FROM "star" WHERE ("star"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,900 [75] [DEBUG] [peewee] ('DELETE FROM "repositorysize" WHERE ("repositorysize"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,904 [75] [DEBUG] [peewee] ('DELETE FROM "repositorysearchscore" WHERE ("repositorysearchscore"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,907 [75] [DEBUG] [peewee] ('DELETE FROM "repositorypermission" WHERE ("repositorypermission"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,908 [75] [DEBUG] [peewee] ('DELETE FROM "repositorynotification" WHERE ("repositorynotification"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,909 [75] [DEBUG] [peewee] ('DELETE FROM "repositoryauthorizedemail" WHERE ("repositoryauthorizedemail"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,909 [75] [DEBUG] [peewee] ('DELETE FROM "repositoryactioncount" WHERE ("repositoryactioncount"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,910 [75] [DEBUG] [peewee] ('DELETE FROM "repomirrorrule" WHERE ("repomirrorrule"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,911 [75] [DEBUG] [peewee] ('DELETE FROM "manifest" WHERE ("manifest"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,912 [75] [DEBUG] [peewee] ('DELETE FROM "image" WHERE ("image"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,913 [75] [DEBUG] [peewee] ('DELETE FROM "deletedrepository" WHERE ("deletedrepository"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,914 [75] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,915 [75] [DEBUG] [peewee] ('DELETE FROM "apprtag" WHERE ("apprtag"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,915 [75] [DEBUG] [peewee] ('DELETE FROM "accesstoken" WHERE ("accesstoken"."repository_id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,916 [75] [DEBUG] [peewee] ('DELETE FROM "repository" WHERE ("repository"."id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,924 [75] [DEBUG] [util.locking] Releasing lock LARGE_GARBAGE_COLLECTION repositorygcworker stdout | 2023-08-08 05:51:09,924 [75] [DEBUG] [redis_lock.release] Releasing 'lock:LARGE_GARBAGE_COLLECTION'. repositorygcworker stdout | 2023-08-08 05:51:09,925 [75] [DEBUG] [util.locking] Released lock LARGE_GARBAGE_COLLECTION repositorygcworker stdout | 2023-08-08 05:51:09,925 [75] [DEBUG] [peewee] ('DELETE FROM "queueitem" WHERE ("queueitem"."id" = %s)', [1]) repositorygcworker stdout | 2023-08-08 05:51:09,929 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 9, 929063), True, datetime.datetime(2023, 8, 8, 5, 51, 9, 929063), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:51:09,931 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:51:09,931 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:51:09,932 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:09 UTC)" executed successfully builder stdout | 2023-08-08 05:51:09,949 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:51:09,958 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:51:09,958 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:51:09,963 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:11,627 [199] [DEBUG] [app] Starting request: urn:request:6f5dbb10-db9d-43b3-91a5-9828fcf4e3a0 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:51:11,627 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:11,627 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:11,628 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:11,635 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:51:11,635 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:51:11,635 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:11,635 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:51:11,635 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:11,635 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:11,635 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:11,635 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:11,638 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:51:11,644 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:51:11,645 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:11,647 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:11,650 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:11,653 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:11,654 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:51:11,655 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:51:11,656 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:51:11,656 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:51:11,656 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:51:11,658 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:51:11,787 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:11,788 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:11 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4986', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '14', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9490:052F:81FF7C:10BF523:64D1D7CF'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAS7CONIMWO4M2D26ZDE2HMPW","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:51:11,890 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/contents/Dockerfile HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:11,891 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:11 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"35a89d24e0791670dc4732e5402e3a749e89c5e6"', 'last-modified': 'Tue, 08 Aug 2023 02:50:20 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4985', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '15', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9490:052F:81FFA3:10BF56C:64D1D7CF'} {"name":"Dockerfile","path":"Dockerfile","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","html_url":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile","git_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","download_url":"https://raw.githubusercontent.com/hu-weihua/quay_build_test/main/Dockerfile?token=ABYHPAWLCDN4EOBF6IHP2ELE2HMAW","type":"file","content":"RlJPTSByZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbS91Ymk3L3ViaQoKVVNF\nUiByb290CkxBQkVMIG1haW50YWluZXI9IndodSIKCiMgVXBkYXRlIGltYWdl\nClJVTiB5dW0gdXBkYXRlIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24t\nbWFuYWdlciAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KUlVOIHl1bSBp\nbnN0YWxsIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24tbWFuYWdlciBo\ndHRwZCAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KCiMgQWRkIGRlZmF1\nbHQgV2ViIHBhZ2UgYW5kIGV4cG9zZSBwb3J0ClJVTiBlY2hvICJUaGUgV2Vi\nIFNlcnZlciBpcyBSdW5uaW5nIiA+IC92YXIvd3d3L2h0bWwvaW5kZXguaHRt\nbApSVU4gZWNobyAicHVzaCB0ZXN0IiA+IC93aHUudHh0ClJVTiBlY2hvICJw\ndXNoIHRlc3QxIiA+IC93aHUudHh0CkVYUE9TRSA4MAoKIyBTdGFydCB0aGUg\nc2VydmljZQpDTUQgWyItRCIsICJGT1JFR1JPVU5EIl0KRU5UUllQT0lOVCBb\nIi91c3Ivc2Jpbi9odHRwZCJdCg==\n","encoding":"base64","_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","git":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","html":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile"}} gunicorn-web stdout | 2023-08-08 05:51:11,894 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 05:51:11,896 [199] [DEBUG] [app] Ending request: urn:request:6f5dbb10-db9d-43b3-91a5-9828fcf4e3a0 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:6f5dbb10-db9d-43b3-91a5-9828fcf4e3a0', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': None}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:51:11,897 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:51:11 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze HTTP/2.0" 200 115 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.272 1116 0.273) gunicorn-web stdout | 2023-08-08 05:51:11,897 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:51:11 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze HTTP/1.0" 200 115 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" chunkcleanupworker stdout | 2023-08-08 05:51:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:51:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:22.293757+00:00 (in 9.999223 seconds) chunkcleanupworker stdout | 2023-08-08 05:51:12,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:22 UTC)" (scheduled at 2023-08-08 05:51:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:51:12,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:51:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 12, 295221), True, datetime.datetime(2023, 8, 8, 5, 51, 12, 295221), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:51:12,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:51:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:51:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:51:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:51:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:31.283850+00:00 (in 18.185818 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:51:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:13 UTC)" (scheduled at 2023-08-08 05:51:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:51:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:51:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 13, 98567), True, datetime.datetime(2023, 8, 8, 5, 51, 13, 98567), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:51:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:51:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:51:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:51:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:51:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:25.371682+00:00 (in 9.999377 seconds) notificationworker stdout | 2023-08-08 05:51:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:25 UTC)" (scheduled at 2023-08-08 05:51:15.371682+00:00) notificationworker stdout | 2023-08-08 05:51:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:51:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 15, 372870), True, datetime.datetime(2023, 8, 8, 5, 51, 15, 372870), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:51:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:51:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:51:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:51:16,322 [200] [DEBUG] [app] Starting request: urn:request:54da2651-3ba7-4f5a-950b-cb029eb71b18 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:51:16,323 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:16,323 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:16,323 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:16,330 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:51:16,330 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:51:16,330 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:16,331 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:51:16,331 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:16,331 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:16,331 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:16,331 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:16,450 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:51:16,457 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:51:16,459 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:16,462 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:16,465 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:16,468 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:16,470 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:51:16,472 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:51:16,472 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:51:16,472 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:51:16,472 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:51:16,475 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:51:16,609 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:16,609 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:16 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4984', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '16', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E098:2B03:EA77A2:1DD0355:64D1D7D4'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAQORZDLZYFG4HXGKFDE2HMQA","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:51:16,692 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/contents/Dockerfile HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:16,693 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:16 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"35a89d24e0791670dc4732e5402e3a749e89c5e6"', 'last-modified': 'Tue, 08 Aug 2023 02:50:20 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4983', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '17', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E098:2B03:EA77C4:1DD0394:64D1D7D4'} {"name":"Dockerfile","path":"Dockerfile","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","html_url":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile","git_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","download_url":"https://raw.githubusercontent.com/hu-weihua/quay_build_test/main/Dockerfile?token=ABYHPAQQ4B57NWHP5KTTTRLE2HMBA","type":"file","content":"RlJPTSByZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbS91Ymk3L3ViaQoKVVNF\nUiByb290CkxBQkVMIG1haW50YWluZXI9IndodSIKCiMgVXBkYXRlIGltYWdl\nClJVTiB5dW0gdXBkYXRlIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24t\nbWFuYWdlciAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KUlVOIHl1bSBp\nbnN0YWxsIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24tbWFuYWdlciBo\ndHRwZCAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KCiMgQWRkIGRlZmF1\nbHQgV2ViIHBhZ2UgYW5kIGV4cG9zZSBwb3J0ClJVTiBlY2hvICJUaGUgV2Vi\nIFNlcnZlciBpcyBSdW5uaW5nIiA+IC92YXIvd3d3L2h0bWwvaW5kZXguaHRt\nbApSVU4gZWNobyAicHVzaCB0ZXN0IiA+IC93aHUudHh0ClJVTiBlY2hvICJw\ndXNoIHRlc3QxIiA+IC93aHUudHh0CkVYUE9TRSA4MAoKIyBTdGFydCB0aGUg\nc2VydmljZQpDTUQgWyItRCIsICJGT1JFR1JPVU5EIl0KRU5UUllQT0lOVCBb\nIi91c3Ivc2Jpbi9odHRwZCJdCg==\n","encoding":"base64","_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","git":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","html":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile"}} gunicorn-web stdout | 2023-08-08 05:51:16,696 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 05:51:16,698 [200] [DEBUG] [app] Ending request: urn:request:54da2651-3ba7-4f5a-950b-cb029eb71b18 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:54da2651-3ba7-4f5a-950b-cb029eb71b18', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:51:16,699 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:51:16 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze HTTP/2.0" 200 115 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.380 552 0.379) gunicorn-web stdout | 2023-08-08 05:51:16,700 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:51:16 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/analyze HTTP/1.0" 200 115 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 05:51:16,970 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:51:18,156 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:51:19,375 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:51:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:51:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:29.417660+00:00 (in 9.999484 seconds) gcworker stdout | 2023-08-08 05:51:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:29 UTC)" (scheduled at 2023-08-08 05:51:19.417660+00:00) gcworker stdout | 2023-08-08 05:51:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264279418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:51:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:51:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:51:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:29 UTC)" executed successfully securityworker stdout | 2023-08-08 05:51:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:51:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:26.890741+00:00 (in 7.202767 seconds) securityworker stdout | 2023-08-08 05:51:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:49 UTC)" (scheduled at 2023-08-08 05:51:19.687363+00:00) securityworker stdout | 2023-08-08 05:51:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:51:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:51:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:51:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:51:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:51:19,700 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:51:19,700 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:49 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:51:19,785 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:51:19,964 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:51:19,973 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:51:19,974 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:51:19,978 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:20,715 [200] [DEBUG] [app] Starting request: urn:request:511011ef-e0b7-40ce-b868-5d582f9c5f5a (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/activate) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:51:20,715 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:20,716 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:20,716 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:20,723 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:51:20,723 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:51:20,723 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:20,724 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:51:20,724 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:20,724 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:20,724 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:20,724 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:51:20,726 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:51:20,733 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:51:20,734 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:20,738 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:20,742 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:20,747 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:20,748 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:51:20,748 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['write', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:20,751 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:20,753 [200] [DEBUG] [peewee] ('INSERT INTO "accesstoken" ("friendly_name", "token_name", "token_code", "repository_id", "created", "role_id", "temporary") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "accesstoken"."id"', ['Build Trigger: github', '3NVXDUTKEMB92FJZTI3ATYFY6ZRV9V63', 'v0$$Ndy2vVtJGAWqOcnPtkBjGa522PdTQStcqdq7o41pReSEw/7OjwjPQC0oEwFhsf2t5yi94mYg3qvPtbe3pg==', 2, datetime.datetime(2023, 8, 8, 5, 51, 20, 752708), 2, False]) gunicorn-web stdout | 2023-08-08 05:51:20,759 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:51:20,878 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:51:20,878 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4982', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '18', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E0A2:33CB:DF6A97:1C75393:64D1D7D8'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAUU3TK6X7XK45NNGD3E2HMQI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:51:21,069 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "POST /repos/hu-weihua/quay_build_test/keys HTTP/1.1" 201 622 gunicorn-web stdout | 2023-08-08 05:51:21,070 [200] [DEBUG] [github.Requester] POST https://api.github.com/repos/hu-weihua/quay_build_test/keys {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python', 'Content-Type': 'application/json'} {"title": "Red Hat Quay Builder", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnLN7FCYWGm7C1durrXhAAKp6NnijqiYjLrPNhw11ohhKjX/DH+qBDJyJBirBGcpMsjRqfxWiVow1fWd6Z2DMa0gruZuhO3c4orINe+LSyCb+RgvEc7UXBSJzhWGeOWR+5Xu3mOOWrc3rESfpV53z8Wo765pdiMoSQnsrTHVRm32pT5FcePiRelkoJTq1VfzCVdI35KEKBimfsVjmG+maqtL0obUQEejbhsJTO656p1zwklUQqKhsHGBbFk+5S8I1hbmAzdnEB4L4UYoGAU1VxrANCU3wRD186xtdAYECNRENp9k4gi0Mz/YiyhH58sfSWh+Nr6MKdYVHxob5tviWZ", "read_only": false} ==> 201 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:21 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '622', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"a065b44f90f0748d6f25ace608b6f11a5ba7219226665f757bd2fe253440ae23"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'location': 'https://api.github.com/repos/hu-weihua/quay_build_test/keys/85930047', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4981', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '19', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': 'E0A2:33CB:DF6AC5:1C753F5:64D1D7D8'} {"id":85930047,"key":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnLN7FCYWGm7C1durrXhAAKp6NnijqiYjLrPNhw11ohhKjX/DH+qBDJyJBirBGcpMsjRqfxWiVow1fWd6Z2DMa0gruZuhO3c4orINe+LSyCb+RgvEc7UXBSJzhWGeOWR+5Xu3mOOWrc3rESfpV53z8Wo765pdiMoSQnsrTHVRm32pT5FcePiRelkoJTq1VfzCVdI35KEKBimfsVjmG+maqtL0obUQEejbhsJTO656p1zwklUQqKhsHGBbFk+5S8I1hbmAzdnEB4L4UYoGAU1VxrANCU3wRD186xtdAYECNRENp9k4gi0Mz/YiyhH58sfSWh+Nr6MKdYVHxob5tviWZ","url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys/85930047","title":"Red Hat Quay Builder","verified":true,"created_at":"2023-08-08T05:51:21Z","read_only":false,"last_used":null,"added_by":"hu-weihua"} gunicorn-web stdout | 2023-08-08 05:51:21,262 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "POST /repos/hu-weihua/quay_build_test/hooks HTTP/1.1" 201 853 gunicorn-web stdout | 2023-08-08 05:51:21,263 [200] [DEBUG] [github.Requester] POST https://api.github.com/repos/hu-weihua/quay_build_test/hooks {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python', 'Content-Type': 'application/json'} {"name": "web", "config": {"url": "https://$token:3NVXDUTKEMB92FJZTI3ATYFY6ZRV9V630RPRD209U09WFR32PEBACK4QHSJ1HEQM@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e", "content_type": "json"}} ==> 201 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:51:21 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '853', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"652d6c78a1e69dffbfc6a80634fa7985667feacedc72e71ffe7ae118f8ef92ac"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:repo_hook, repo, write:repo_hook', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'location': 'https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4980', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '20', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': 'E0A2:33CB:DF6B16:1C7548E:64D1D7D9'} {"type":"Repository","id":427817209,"name":"web","active":true,"events":["push"],"config":{"content_type":"json","insecure_ssl":"0","url":"https://$token:3NVXDUTKEMB92FJZTI3ATYFY6ZRV9V630RPRD209U09WFR32PEBACK4QHSJ1HEQM@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e"},"updated_at":"2023-08-08T05:51:21Z","created_at":"2023-08-08T05:51:21Z","url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209","test_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209/test","ping_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209/pings","deliveries_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209/deliveries","last_response":{"code":null,"status":"unused","message":null}} gunicorn-web stdout | 2023-08-08 05:51:21,265 [200] [DEBUG] [peewee] ('UPDATE "repositorybuildtrigger" SET "uuid" = %s, "service_id" = %s, "repository_id" = %s, "connected_user_id" = %s, "secure_auth_token" = %s, "secure_private_key" = %s, "fully_migrated" = %s, "config" = %s, "write_token_id" = %s, "pull_robot_id" = %s, "enabled" = %s, "disabled_reason_id" = %s, "disabled_datetime" = %s, "successive_failure_count" = %s, "successive_internal_error_count" = %s WHERE ("repositorybuildtrigger"."id" = %s)', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 2, 1, 'v0$$ZXz9fg6IfOmmIjqfVObhiTlHx0Fdkz+KTywlswvlUP5eVk7rIlIqy/AF6QFZwc7GZWA467JrklqJMFKY0e+QMc3DqcQT', 'v0$$t1x7fx2e2G2aaZ+aGyzw3wU9DCRAUIc1whFdhAJKmJ0cK3LbgJ6yadvOw5kl7n2QybQ0JgS5IpxZ0b9h3oUltUwVKBc5D1O/cnIe7FOAriMSLdDZ0AHivBM3vfBPM66nUmGmvAOKFCRJJy+fvntvrsk5P/m1GDuGxLLff3bVftnb/oFra9QZj6kMCW6WgisK4owlrzqDjyF3CZ+8+5ngurP17LHyans2y4D06hXvOTgckWUZYcnV6sUj5Vj4NpbNDent+Q62lcmsioYr+zGKpNj7ab7txbiMo9vgupXLsTaIGr5kO9T2cw9FS7M9vbYYdpWDe+2/UzGTbsUqRS8XWxd8iiAq4X4M8gmi3blBwnWW1fLLk6vYfXFgApT7kajj00kUldO9rXNIOWL4t8TV/pf89zdaGGhjgcGplvYqv4HNt6h5zjDEdlfHtJbwCcF2O3QiJB3FiaJwxkmwfEq5hOmG769/kZlMiguG3o7IMv9o2GMEFLkV0hcq3WPdyP10X6YZQkDkvA3v80t6sL8sQOcwNrw+Do9ueXUzCnJsgLRmFpA5sTam8NjUJ2eiO2SKxdZ2eEVCngCGhQUoPNG7rVceDSld/+amBtqgw+VVXpL1Mx4S2oP1+nvkFrHD7Q95z0t8GAT4Cf6nnchZ35QU9P5CDfbYFy9KUQMMKfbE6xwqNDSLAYg5uBmEJIx7Z4HPJ2gIpIYMYyrC9PT/htqzjnIpxrEvBzWvPEGlBQgN15FdKmsrBVmJHP/3K/8YgvUoafRCRr2s6yDfzBpDFSIzhnhCbWkus5fzdmpo34jI3ebQylR/thxeEg2UdZycsM+SFlioTkdgtYtjp+5gYxZK2NCA2VaHkBM/Y0343/E3NBq8S+kS+QW8PSlS6Au8c6f33LEwhiAey9MYM4PbwixMqXbRUTBIF6svroycVSbqSNYnmrBl/iJse0gF5tWw7UCfeW6Gy4Lp/P23yUOrOXJONFkflv6Lyoxvwe7VP4NqvEdUY8DMZ7f0dmji00CANWbw5m9P2aTm145AV7+2LKAXXH06OVzDTE23zgHNtpxGUUB/2CFwK/DalYUy8Pc4HyJZQ2PbPxg2tPkNNj+ZJ2BXIPvYnLMQpanc9PfqT4lb5NRLKacmptxwJxoLI4Py8c0DR1+tXaKVkYgKSxv6gh0k5xNGep1fewKd6l+k4tf3ua8qeFY2l/wN9aV7GrX5Ll6L1OA916y5LY3OhqgRpvyp6rUF4cHv+VuDh6x8f0wxLq5nUBngGKdaG37qdaA8P++plGZTMLt0hhH66O0axHKaWohZkyXyNzZw+w3MzxkarUV0eiyW9GB74dlkzu3NKUpfcX1ickxi4NxJSV/E6yx7HfWympDX0HyVutOFePlKvBrCq31VA1C4dMm3Fu+Evd3BvtJQno+AlA4TMZiZ7LtUkROy2MOZw3UM+r1D52GjN82V1SllXwU7088XFUZ25KECqHf8KZM2TQbifiqBYODORN3TuA4izi5Z2SoNHpwXSWk+V9+hIKWB5db0MYwSLsKrUo6E6YXVtbvP/6UEmbWhdzPDDY9WWY3PNb3lgkIWulpOVUr+/YIglBGnyO2Ly9srcozs+l/qUzU1+pQGbsWLHSDXA4okoJCBNx5uPyy5esg1gWmB/3qgFnVIndXlu+o0uOBRUnVERUwY/stLc7amx3J8Y8dwWAqVnFyofS7l11WCOoJyERVO0S9YKv4vBDbS+al+on/nB9JMIBAfanK7Dx7CYJjvz/+AEB3rDuvg4DEy4xCoJDxhcwQAJ9fxbAusAtCi3K9ClKQ+Xf8XrRGZBgKlfcC1d40n/aSwK91XLtk6f+kDU9zHCaTpkZg3VI/IuY7NoxBNGYYFtwTn0MM4j0zQb7HNTgYABUkcxYFp64GsbYXO8DJmiMFpAZSRBEy9yVzx7+hoHdMR6flcFeFOSZnByCemKUWUqNTqAZ5QPEi8sw/Mhsc91t3fWlXElFfpTOHn+VsVauUaZKEmAJ4jCB11SdwrpcTqK8tTjUXAE/YhzrChSK/eqjp30Y/e5gjllIcywv4phCE2LkksIjS3Zll0POWYxHMIgVBWbGRoeCS9QW1UXzbQQDCqwei92P7bbTk1Qi/5Nyz+k627iSG7GeTNTgbxiFxIxA2pKmTtg9rdolHJ9sn1bG2htWQJ8UCcaPXU0bKMcDUywt62AW5Z0w/45V07hwC/ltrVuDW2qnQnuMTJ2ppUL9gI0+Ctkj2k3DSnyEwiKXqrEH3LrZDIM/onPRaX0kRa', False, '{"build_source": "hu-weihua/quay_build_test", "dockerfile_path": "/Dockerfile", "context": "/", "default_tag_from_ref": true, "latest_for_default_branch": true, "tag_templates": [], "credentials": [{"name": "SSH Public Key", "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnLN7FCYWGm7C1durrXhAAKp6NnijqiYjLrPNhw11ohhKjX/DH+qBDJyJBirBGcpMsjRqfxWiVow1fWd6Z2DMa0gruZuhO3c4orINe+LSyCb+RgvEc7UXBSJzhWGeOWR+5Xu3mOOWrc3rESfpV53z8Wo765pdiMoSQnsrTHVRm32pT5FcePiRelkoJTq1VfzCVdI35KEKBimfsVjmG+maqtL0obUQEejbhsJTO656p1zwklUQqKhsHGBbFk+5S8I1hbmAzdnEB4L4UYoGAU1VxrANCU3wRD186xtdAYECNRENp9k4gi0Mz/YiyhH58sfSWh+Nr6MKdYVHxob5tviWZ"}], "deploy_key_id": 85930047, "hook_id": 427817209, "master_branch": "main"}', 1, None, True, None, datetime.datetime(2023, 8, 8, 5, 50, 57, 51804), 0, 0, 1]) gunicorn-web stdout | 2023-08-08 05:51:21,270 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:21,272 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:21,273 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "logentrykind" AS "t1"', []) gunicorn-web stdout | 2023-08-08 05:51:21,276 [200] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [53, 2, 1, 2, datetime.datetime(2023, 8, 8, 5, 51, 21, 271904), '10.130.2.1', '{"repo": "github", "namespace": "build_test", "trigger_id": "3f8c84f2-7889-4e84-8777-d2fe1e46ea1e", "service": "github", "pull_robot": null, "config": {"build_source": "hu-weihua/quay_build_test", "dockerfile_path": "/Dockerfile", "context": "/", "default_tag_from_ref": true, "latest_for_default_branch": true, "tag_templates": [], "credentials": [{"name": "SSH Public Key", "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnLN7FCYWGm7C1durrXhAAKp6NnijqiYjLrPNhw11ohhKjX/DH+qBDJyJBirBGcpMsjRqfxWiVow1fWd6Z2DMa0gruZuhO3c4orINe+LSyCb+RgvEc7UXBSJzhWGeOWR+5Xu3mOOWrc3rESfpV53z8Wo765pdiMoSQnsrTHVRm32pT5FcePiRelkoJTq1VfzCVdI35KEKBimfsVjmG+maqtL0obUQEejbhsJTO656p1zwklUQqKhsHGBbFk+5S8I1hbmAzdnEB4L4UYoGAU1VxrANCU3wRD186xtdAYECNRENp9k4gi0Mz/YiyhH58sfSWh+Nr6MKdYVHxob5tviWZ"}], "deploy_key_id": 85930047, "hook_id": 427817209, "master_branch": "main"}}']) gunicorn-web stdout | 2023-08-08 05:51:21,279 [200] [DEBUG] [app] Ending request: urn:request:511011ef-e0b7-40ce-b868-5d582f9c5f5a (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/activate) {'endpoint': 'api.buildtriggeractivate', 'request_id': 'urn:request:511011ef-e0b7-40ce-b868-5d582f9c5f5a', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/activate', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/activate', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': '/Dockerfile', 'context': '/', 'default_tag_from_ref': True, 'latest_for_default_branch': True, 'tag_templates': [], 'credentials': [{'name': 'SSH Public Key', 'value': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnLN7FCYWGm7C1durrXhAAKp6NnijqiYjLrPNhw11ohhKjX/DH+qBDJyJBirBGcpMsjRqfxWiVow1fWd6Z2DMa0gruZuhO3c4orINe+LSyCb+RgvEc7UXBSJzhWGeOWR+5Xu3mOOWrc3rESfpV53z8Wo765pdiMoSQnsrTHVRm32pT5FcePiRelkoJTq1VfzCVdI35KEKBimfsVjmG+maqtL0obUQEejbhsJTO656p1zwklUQqKhsHGBbFk+5S8I1hbmAzdnEB4L4UYoGAU1VxrANCU3wRD186xtdAYECNRENp9k4gi0Mz/YiyhH58sfSWh+Nr6MKdYVHxob5tviWZ'}], 'deploy_key_id': 85930047, 'hook_id': 427817209, 'master_branch': 'main'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:51:21,280 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:51:21 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/activate HTTP/2.0" 200 965 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.567 632 0.566) gunicorn-web stdout | 2023-08-08 05:51:21,280 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:51:21 +0000] "POST /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/activate HTTP/1.0" 200 965 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:51:22,179 [199] [DEBUG] [app] Starting request: urn:request:517516e3-ff31-4045-b6f8-0fc32ab41831 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:51:22,179 [197] [DEBUG] [app] Starting request: urn:request:a9cc5051-cb81-4b8d-bb92-06b205032b10 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:51:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:22,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:51:22,185 [208] [DEBUG] [app] Starting request: urn:request:8c4ffd50-0405-4e62-aa4d-aeaf62b2b8c5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:51:22,186 [208] [DEBUG] [app] Ending request: urn:request:8c4ffd50-0405-4e62-aa4d-aeaf62b2b8c5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8c4ffd50-0405-4e62-aa4d-aeaf62b2b8c5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:51:22,186 [210] [DEBUG] [app] Starting request: urn:request:06d1d62c-e5b3-4b28-89e9-64d3e8551d12 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:51:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:51:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:51:22,186 [210] [DEBUG] [app] Ending request: urn:request:06d1d62c-e5b3-4b28-89e9-64d3e8551d12 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:06d1d62c-e5b3-4b28-89e9-64d3e8551d12', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:51:22,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:51:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:22,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:22,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:22,192 [198] [DEBUG] [app] Starting request: urn:request:0a883bf1-0a7c-4ef2-a9f4-ac3610659dd1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:51:22,192 [200] [DEBUG] [app] Starting request: urn:request:7a6c1a3a-e39d-45d3-97da-38529fe9779e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:51:22,193 [198] [DEBUG] [app] Ending request: urn:request:0a883bf1-0a7c-4ef2-a9f4-ac3610659dd1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0a883bf1-0a7c-4ef2-a9f4-ac3610659dd1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:51:22,193 [200] [DEBUG] [app] Ending request: urn:request:7a6c1a3a-e39d-45d3-97da-38529fe9779e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7a6c1a3a-e39d-45d3-97da-38529fe9779e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:51:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:22,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:51:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:22,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:51:22,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:51:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:51:22,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:51:22,204 [197] [DEBUG] [app] Ending request: urn:request:a9cc5051-cb81-4b8d-bb92-06b205032b10 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a9cc5051-cb81-4b8d-bb92-06b205032b10', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:51:22,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:22,205 [199] [DEBUG] [app] Ending request: urn:request:517516e3-ff31-4045-b6f8-0fc32ab41831 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:517516e3-ff31-4045-b6f8-0fc32ab41831', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:51:22,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:51:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:51:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:51:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:51:22,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:51:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:51:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:51:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:28.874906+00:00 (in 6.580332 seconds) chunkcleanupworker stdout | 2023-08-08 05:51:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:32 UTC)" (scheduled at 2023-08-08 05:51:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:51:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:51:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 22, 294953), True, datetime.datetime(2023, 8, 8, 5, 51, 22, 294953), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:51:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:51:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:51:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:51:22,577 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:51:24,387 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:51:24,467 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:51:24,692 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:51:24,695 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:51:24,698 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:51:24,707 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:51:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:51:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:35.371682+00:00 (in 9.999412 seconds) notificationworker stdout | 2023-08-08 05:51:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:35 UTC)" (scheduled at 2023-08-08 05:51:25.371682+00:00) notificationworker stdout | 2023-08-08 05:51:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:51:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 25, 372804), True, datetime.datetime(2023, 8, 8, 5, 51, 25, 372804), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:51:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:51:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:51:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:51:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:51:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:47.398551+00:00 (in 21.883623 seconds) exportactionlogsworker stdout | 2023-08-08 05:51:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:25 UTC)" (scheduled at 2023-08-08 05:51:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:51:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:51:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:51:25,812 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:51:26,393 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:51:26,584 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:51:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:51:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:49.687363+00:00 (in 22.796100 seconds) securityworker stdout | 2023-08-08 05:51:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:56 UTC)" (scheduled at 2023-08-08 05:51:26.890741+00:00) securityworker stdout | 2023-08-08 05:51:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:51:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:51:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:51:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:51:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:51:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:51:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:51:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:51:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:51:26,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:51:26,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:51:26,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:51:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:51:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:51:26,904 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:51:26,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:51:26,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:51:26,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:51:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:51:27,088 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:51:27,090 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:51:27,096 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:51:27,102 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:51:27,104 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:51:27,765 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:51:27,863 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:51:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:51:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:32.293757+00:00 (in 3.418331 seconds) chunkcleanupworker stdout | 2023-08-08 05:51:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:28 UTC)" (scheduled at 2023-08-08 05:51:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:51:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:51:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:51:28,990 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:51:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:51:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:39.417660+00:00 (in 9.999491 seconds) gcworker stdout | 2023-08-08 05:51:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:39 UTC)" (scheduled at 2023-08-08 05:51:29.417660+00:00) gcworker stdout | 2023-08-08 05:51:29,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264289418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:51:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:51:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:51:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:51:29,830 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:51:29,891 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:51:29,988 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:51:29,998 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:51:29,998 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:51:30,002 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:51:30,403 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:51:30,951 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:51:31,253 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} securityscanningnotificationworker stdout | 2023-08-08 05:51:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:51:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:54.284093+00:00 (in 22.999650 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:51:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:56:31 UTC)" (scheduled at 2023-08-08 05:51:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:51:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 51, 31, 284939), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:51:31,293 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 51, 31, 284939), True, datetime.datetime(2023, 8, 8, 5, 51, 31, 284939), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:51:31,296 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 51, 31, 284939), True, datetime.datetime(2023, 8, 8, 5, 51, 31, 284939), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 5, 51, 31, 284939), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:51:31,297 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:51:31,297 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:56:31 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 05:51:31,523 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:51:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:51:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:42.293757+00:00 (in 9.999362 seconds) chunkcleanupworker stdout | 2023-08-08 05:51:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:42 UTC)" (scheduled at 2023-08-08 05:51:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:51:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:51:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 32, 294926), True, datetime.datetime(2023, 8, 8, 5, 51, 32, 294926), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:51:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:51:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:51:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:51:32,886 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:51:32,894 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:51:32,897 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:51:32,899 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:51:32,907 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:51:32,909 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:51:32,913 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:51:32,916 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:51:32,919 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 05:51:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:51:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:45.371682+00:00 (in 9.999446 seconds) notificationworker stdout | 2023-08-08 05:51:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:45 UTC)" (scheduled at 2023-08-08 05:51:35.371682+00:00) notificationworker stdout | 2023-08-08 05:51:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:51:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 35, 372821), True, datetime.datetime(2023, 8, 8, 5, 51, 35, 372821), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:51:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:51:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:51:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:51:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:51:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:54.393627+00:00 (in 17.813266 seconds) namespacegcworker stdout | 2023-08-08 05:51:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:36 UTC)" (scheduled at 2023-08-08 05:51:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:51:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:51:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:51:37,180 [199] [DEBUG] [app] Starting request: urn:request:8ba56d41-76a9-46ba-8ad1-0f92c57d972e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:51:37,180 [197] [DEBUG] [app] Starting request: urn:request:48e5f8eb-c834-49b6-8c95-3547eea83d5b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:51:37,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:37,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:51:37,186 [203] [DEBUG] [app] Starting request: urn:request:9d80e711-199f-4cca-b9e2-8fb4b031c422 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:51:37,186 [203] [DEBUG] [app] Ending request: urn:request:9d80e711-199f-4cca-b9e2-8fb4b031c422 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9d80e711-199f-4cca-b9e2-8fb4b031c422', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:51:37,186 [208] [DEBUG] [app] Starting request: urn:request:2c28a60d-4ba9-4042-983f-c6b904ed8ed2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:51:37,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:51:37,187 [208] [DEBUG] [app] Ending request: urn:request:2c28a60d-4ba9-4042-983f-c6b904ed8ed2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2c28a60d-4ba9-4042-983f-c6b904ed8ed2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:51:37,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:37,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:37,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:37,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:37,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:37,192 [197] [DEBUG] [app] Starting request: urn:request:d0eb0c3a-888e-41f9-bc97-0cfeb01cfe7a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:51:37,193 [197] [DEBUG] [app] Ending request: urn:request:d0eb0c3a-888e-41f9-bc97-0cfeb01cfe7a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d0eb0c3a-888e-41f9-bc97-0cfeb01cfe7a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:51:37,193 [198] [DEBUG] [app] Starting request: urn:request:6ce95180-6ee7-4f61-9ce5-4e36713405cd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:51:37,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:37,194 [198] [DEBUG] [app] Ending request: urn:request:6ce95180-6ee7-4f61-9ce5-4e36713405cd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6ce95180-6ee7-4f61-9ce5-4e36713405cd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:51:37,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:51:37,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:37,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:37,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:51:37,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:51:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:51:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:07.197412+00:00 (in 29.999487 seconds) buildlogsarchiver stdout | 2023-08-08 05:51:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:07 UTC)" (scheduled at 2023-08-08 05:51:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:51:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 51, 37, 198415), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:51:37,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:51:37,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:51:37,205 [197] [DEBUG] [app] Ending request: urn:request:48e5f8eb-c834-49b6-8c95-3547eea83d5b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:48e5f8eb-c834-49b6-8c95-3547eea83d5b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:51:37,205 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:51:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 05:51:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:51:37,205 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:51:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:51:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:51:37,206 [199] [DEBUG] [app] Ending request: urn:request:8ba56d41-76a9-46ba-8ad1-0f92c57d972e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8ba56d41-76a9-46ba-8ad1-0f92c57d972e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:51:37,206 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:51:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) gunicorn-web stdout | 2023-08-08 05:51:37,206 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:51:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gcworker stdout | 2023-08-08 05:51:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:51:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:49.417660+00:00 (in 9.999469 seconds) gcworker stdout | 2023-08-08 05:51:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:49 UTC)" (scheduled at 2023-08-08 05:51:39.417660+00:00) gcworker stdout | 2023-08-08 05:51:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264299418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:51:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:51:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:51:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:49 UTC)" executed successfully builder stdout | 2023-08-08 05:51:40,013 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:51:40,023 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:51:40,023 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:51:40,029 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:51:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:51:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:52.293757+00:00 (in 9.999376 seconds) chunkcleanupworker stdout | 2023-08-08 05:51:42,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:52 UTC)" (scheduled at 2023-08-08 05:51:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:51:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:51:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 42, 295368), True, datetime.datetime(2023, 8, 8, 5, 51, 42, 295368), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:51:42,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:51:42,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:51:42,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:51:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:51:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:55.371682+00:00 (in 9.999004 seconds) notificationworker stdout | 2023-08-08 05:51:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:55 UTC)" (scheduled at 2023-08-08 05:51:45.371682+00:00) notificationworker stdout | 2023-08-08 05:51:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:51:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 45, 373107), True, datetime.datetime(2023, 8, 8, 5, 51, 45, 373107), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:51:45,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:51:45,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:51:45,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:51:47,008 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:51:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:51:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:25.514341+00:00 (in 38.115243 seconds) exportactionlogsworker stdout | 2023-08-08 05:51:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:47 UTC)" (scheduled at 2023-08-08 05:51:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:51:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:51:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 47, 399601), True, datetime.datetime(2023, 8, 8, 5, 51, 47, 399601), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:51:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:51:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:51:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:51:48,177 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 05:51:49,412 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:51:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:51:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:59.417660+00:00 (in 9.999473 seconds) gcworker stdout | 2023-08-08 05:51:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:59 UTC)" (scheduled at 2023-08-08 05:51:49.417660+00:00) gcworker stdout | 2023-08-08 05:51:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264309418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:51:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:51:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:51:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:51:59 UTC)" executed successfully securityworker stdout | 2023-08-08 05:51:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:51:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:51:56.890741+00:00 (in 7.202780 seconds) securityworker stdout | 2023-08-08 05:51:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:19 UTC)" (scheduled at 2023-08-08 05:51:49.687363+00:00) securityworker stdout | 2023-08-08 05:51:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:51:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:51:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:51:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:51:49,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:51:49,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:51:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:19 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:51:49,821 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:51:50,039 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:51:50,048 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:51:50,049 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:51:50,053 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:52,178 [200] [DEBUG] [app] Starting request: urn:request:8504d23e-b025-42d2-85f9-2ec19e9ab8a7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:51:52,180 [197] [DEBUG] [app] Starting request: urn:request:b3847699-3202-451b-a1d1-56b39a6183c8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:51:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:51:52,184 [208] [DEBUG] [app] Starting request: urn:request:4a466f3f-bd7c-4f0a-8d40-eed8d7d1dda0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:51:52,185 [209] [DEBUG] [app] Starting request: urn:request:63dcb504-dffe-445e-9d3a-5b1030141a63 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:51:52,185 [208] [DEBUG] [app] Ending request: urn:request:4a466f3f-bd7c-4f0a-8d40-eed8d7d1dda0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4a466f3f-bd7c-4f0a-8d40-eed8d7d1dda0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:51:52,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:51:52,185 [209] [DEBUG] [app] Ending request: urn:request:63dcb504-dffe-445e-9d3a-5b1030141a63 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:63dcb504-dffe-445e-9d3a-5b1030141a63', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:51:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:51:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:51:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:51:52,190 [200] [DEBUG] [app] Starting request: urn:request:63d3e6c6-cf3b-4793-a183-1abee62f9761 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:51:52,191 [200] [DEBUG] [app] Ending request: urn:request:63d3e6c6-cf3b-4793-a183-1abee62f9761 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:63d3e6c6-cf3b-4793-a183-1abee62f9761', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 05:51:52,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:52,191 [199] [DEBUG] [app] Starting request: urn:request:35c0ed94-1c73-4c75-bb66-e9137562e819 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:51:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:52,192 [199] [DEBUG] [app] Ending request: urn:request:35c0ed94-1c73-4c75-bb66-e9137562e819 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:35c0ed94-1c73-4c75-bb66-e9137562e819', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:51:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:51:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:51:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:51:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:51:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:51:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:51:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:51:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:51:52,204 [200] [DEBUG] [app] Ending request: urn:request:8504d23e-b025-42d2-85f9-2ec19e9ab8a7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8504d23e-b025-42d2-85f9-2ec19e9ab8a7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:51:52,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:51:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 05:51:52,204 [197] [DEBUG] [app] Ending request: urn:request:b3847699-3202-451b-a1d1-56b39a6183c8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b3847699-3202-451b-a1d1-56b39a6183c8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:51:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:51:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:51:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:51:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:51:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:51:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) chunkcleanupworker stdout | 2023-08-08 05:51:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:51:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:02.293757+00:00 (in 9.999372 seconds) chunkcleanupworker stdout | 2023-08-08 05:51:52,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:02 UTC)" (scheduled at 2023-08-08 05:51:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:51:52,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:51:52,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 52, 295600), True, datetime.datetime(2023, 8, 8, 5, 51, 52, 295600), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:51:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:51:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:51:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:51:52,608 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:51:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:51:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:13.097375+00:00 (in 18.812663 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:51:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:54 UTC)" (scheduled at 2023-08-08 05:51:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:51:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:51:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:51:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:51:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:36.579841+00:00 (in 42.185640 seconds) namespacegcworker stdout | 2023-08-08 05:51:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:54 UTC)" (scheduled at 2023-08-08 05:51:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:51:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:51:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 54, 394764), True, datetime.datetime(2023, 8, 8, 5, 51, 54, 394764), 0, 'namespacegc/%', 50, 1, 0]) blobuploadcleanupworker stdout | 2023-08-08 05:51:54,397 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} namespacegcworker stdout | 2023-08-08 05:51:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:51:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:51:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:52:54 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:51:54,474 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:51:54,700 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:51:54,702 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:51:54,705 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:51:54,741 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:51:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:51:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:02.178530+00:00 (in 6.806313 seconds) notificationworker stdout | 2023-08-08 05:51:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:05 UTC)" (scheduled at 2023-08-08 05:51:55.371682+00:00) notificationworker stdout | 2023-08-08 05:51:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:51:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 51, 55, 372597), True, datetime.datetime(2023, 8, 8, 5, 51, 55, 372597), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:51:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:51:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:51:55,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:51:55,849 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:51:56,406 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:51:56,606 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:51:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:51:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:19.687363+00:00 (in 22.796081 seconds) securityworker stdout | 2023-08-08 05:51:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:26 UTC)" (scheduled at 2023-08-08 05:51:56.890741+00:00) securityworker stdout | 2023-08-08 05:51:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:51:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:51:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:51:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:51:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:51:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:51:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:51:56,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:51:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:51:56,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:51:56,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:51:56,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:51:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:51:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:51:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:51:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:51:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:51:56,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:51:57,100 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:51:57,103 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:51:57,106 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:51:57,113 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:51:57,116 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:51:57,803 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:51:57,885 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:51:59,014 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:51:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:51:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:09.417660+00:00 (in 9.999441 seconds) gcworker stdout | 2023-08-08 05:51:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:09 UTC)" (scheduled at 2023-08-08 05:51:59.417660+00:00) gcworker stdout | 2023-08-08 05:51:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264319418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:51:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:51:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:51:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:51:59,838 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:51:59,926 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:52:00,063 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:52:00,081 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:52:00,081 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:52:00,087 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:52:00,441 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:52:00,981 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:52:01,277 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:52:01,538 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:52:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:52:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:05.371682+00:00 (in 3.192634 seconds) notificationworker stdout | 2023-08-08 05:52:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:02 UTC)" (scheduled at 2023-08-08 05:52:02.178530+00:00) notificationworker stdout | 2023-08-08 05:52:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:52:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:52:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:52:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:12.293757+00:00 (in 9.999342 seconds) chunkcleanupworker stdout | 2023-08-08 05:52:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:12 UTC)" (scheduled at 2023-08-08 05:52:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:52:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:52:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 2, 295026), True, datetime.datetime(2023, 8, 8, 5, 52, 2, 295026), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:52:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:52:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:52:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:52:02,902 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:52:02,904 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:52:02,907 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:52:02,911 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:52:02,914 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:52:02,919 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:52:02,923 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:52:02,927 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:52:02,929 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:52:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:52:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:09.793658+00:00 (in 5.605637 seconds) repositorygcworker stdout | 2023-08-08 05:52:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:04 UTC)" (scheduled at 2023-08-08 05:52:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:52:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:52:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:52:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:52:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:15.371682+00:00 (in 9.999473 seconds) notificationworker stdout | 2023-08-08 05:52:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:15 UTC)" (scheduled at 2023-08-08 05:52:05.371682+00:00) notificationworker stdout | 2023-08-08 05:52:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:52:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 5, 372640), True, datetime.datetime(2023, 8, 8, 5, 52, 5, 372640), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:52:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:52:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:52:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:05,503 [200] [DEBUG] [app] Starting request: urn:request:db686740-3fb5-4cc9-b73a-fe8711cfd665 (/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:05,512 [200] [DEBUG] [endpoints.api.discovery] Unable to find method for POST in class gunicorn-web stdout | 2023-08-08 05:52:05,517 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:05,538 [200] [DEBUG] [app] Ending request: urn:request:db686740-3fb5-4cc9-b73a-fe8711cfd665 (/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'endpoint': 'web.buildtrigger', 'request_id': 'urn:request:db686740-3fb5-4cc9-b73a-fe8711cfd665', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'path': '/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:05,540 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:05 +0000] "GET /repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/1.0" 200 85192 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:05 +0000] "GET /repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/2.0" 200 13315 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.038 847 0.038) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:06 +0000] "GET /static/build/main-quay-frontend-ab93c42f539d59de0115.bundle.js HTTP/2.0" 200 719930 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.057 558 -) gunicorn-web stdout | 2023-08-08 05:52:07,180 [200] [DEBUG] [app] Starting request: urn:request:ef4dd5e3-3b9e-4eaa-b365-fde4474f6d13 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:52:07,180 [199] [DEBUG] [app] Starting request: urn:request:a5e129d1-5e24-48b8-b88a-f0eef5229a59 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:52:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:07,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:07,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:52:07,185 [208] [DEBUG] [app] Starting request: urn:request:722915f6-31b3-4a8b-9d93-13d3e0fd71c0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:52:07,186 [208] [DEBUG] [app] Ending request: urn:request:722915f6-31b3-4a8b-9d93-13d3e0fd71c0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:722915f6-31b3-4a8b-9d93-13d3e0fd71c0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:52:07,186 [203] [DEBUG] [app] Starting request: urn:request:1b2a8947-d830-4b6e-830f-96f9ff6fe170 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:52:07,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:52:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:52:07,187 [203] [DEBUG] [app] Ending request: urn:request:1b2a8947-d830-4b6e-830f-96f9ff6fe170 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1b2a8947-d830-4b6e-830f-96f9ff6fe170', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:52:07,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 05:52:07,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:07,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:07,191 [197] [DEBUG] [app] Starting request: urn:request:f27aa28e-030e-4955-b043-c19cd0d7f861 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:52:07,192 [197] [DEBUG] [app] Ending request: urn:request:f27aa28e-030e-4955-b043-c19cd0d7f861 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f27aa28e-030e-4955-b043-c19cd0d7f861', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:52:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:52:07,192 [198] [DEBUG] [app] Starting request: urn:request:d969e1f9-f080-43b5-89a1-bf18ab3866c3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:52:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:07,193 [198] [DEBUG] [app] Ending request: urn:request:d969e1f9-f080-43b5-89a1-bf18ab3866c3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d969e1f9-f080-43b5-89a1-bf18ab3866c3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:52:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:52:07,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:52:07,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:52:07,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:52:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 05:52:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:52:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:37.197412+00:00 (in 29.999488 seconds) buildlogsarchiver stdout | 2023-08-08 05:52:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:37 UTC)" (scheduled at 2023-08-08 05:52:07.197412+00:00) gunicorn-web stdout | 2023-08-08 05:52:07,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:52:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 52, 7, 198419), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:07,203 [200] [DEBUG] [app] Ending request: urn:request:ef4dd5e3-3b9e-4eaa-b365-fde4474f6d13 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ef4dd5e3-3b9e-4eaa-b365-fde4474f6d13', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:52:07,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:52:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:52:07,204 [199] [DEBUG] [app] Ending request: urn:request:a5e129d1-5e24-48b8-b88a-f0eef5229a59 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a5e129d1-5e24-48b8-b88a-f0eef5229a59', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:52:07,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:07,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:52:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:52:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:52:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) buildlogsarchiver stdout | 2023-08-08 05:52:07,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:52:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:52:07,207 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:37 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:07 +0000] "GET /static/img/quay_favicon.png HTTP/2.0" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 81 -) gunicorn-web stdout | 2023-08-08 05:52:08,251 [197] [DEBUG] [app] Starting request: urn:request:0ee4cab4-ab9e-48d0-8646-9f534a89a5d9 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:08,252 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,252 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,252 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,259 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:08,259 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:08,259 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,260 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,260 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,260 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,260 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,261 [197] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 05:52:08,263 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,264 [197] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 05:52:08,265 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:52:08,267 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,268 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:52:08,270 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,271 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691473928270, 0, 1, None]) gunicorn-web stdout | 2023-08-08 05:52:08,275 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,276 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:52:08,277 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,279 [197] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691473928278, 0, 1]) gunicorn-web stdout | 2023-08-08 05:52:08,279 [199] [DEBUG] [app] Starting request: urn:request:a67d6418-6a44-4ab2-896a-b6c8ef28ca41 (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:08,280 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,280 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,280 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,280 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,280 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,281 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:52:08,283 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,283 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,283 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,283 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:52:08,283 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,283 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,283 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,284 [197] [DEBUG] [app] Ending request: urn:request:0ee4cab4-ab9e-48d0-8646-9f534a89a5d9 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:0ee4cab4-ab9e-48d0-8646-9f534a89a5d9', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:08,285 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/user/ HTTP/2.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.035 171 0.035) gunicorn-web stdout | 2023-08-08 05:52:08,285 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/user/ HTTP/1.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:52:08,286 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:08,287 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:08,287 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,287 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 05:52:08,289 [199] [DEBUG] [app] Ending request: urn:request:a67d6418-6a44-4ab2-896a-b6c8ef28ca41 (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:a67d6418-6a44-4ab2-896a-b6c8ef28ca41', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:08,290 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 47 0.012) gunicorn-web stdout | 2023-08-08 05:52:08,290 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:52:08,351 [197] [DEBUG] [app] Starting request: urn:request:7ef47199-88e6-43f3-be2b-d1b2bced7c1c (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:08,351 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,351 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,351 [199] [DEBUG] [app] Starting request: urn:request:091cb253-e521-41e9-89ad-6ed586c7bc87 (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:08,351 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,351 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,352 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,352 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,357 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:08,358 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:08,358 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:08,358 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,358 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:08,358 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,358 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:08,358 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,358 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,358 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,358 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,358 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:08,358 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,358 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,358 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,358 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,360 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:08,361 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:08,367 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,367 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,367 [199] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:08,368 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,368 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,369 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [2, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,371 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [2, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:52:08,371 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,372 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,372 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,372 [199] [DEBUG] [app] Ending request: urn:request:091cb253-e521-41e9-89ad-6ed586c7bc87 (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:091cb253-e521-41e9-89ad-6ed586c7bc87', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github?includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:08,373 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/2.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.023 75 0.024) gunicorn-web stdout | 2023-08-08 05:52:08,374 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/1.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:52:08,374 [197] [DEBUG] [app] Ending request: urn:request:7ef47199-88e6-43f3-be2b-d1b2bced7c1c (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:7ef47199-88e6-43f3-be2b-d1b2bced7c1c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:08,374 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/2.0" 200 965 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 94 0.026) gunicorn-web stdout | 2023-08-08 05:52:08,375 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/1.0" 200 965 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:52:08,602 [200] [DEBUG] [app] Starting request: urn:request:b2151f2e-d8cd-4cd9-bede-4f2be5348153 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:08,602 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,602 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,603 [198] [DEBUG] [app] Starting request: urn:request:165652b5-f138-40f8-8fa3-18e73a2101fe (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:08,603 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,603 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,603 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,604 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,609 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:08,609 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:08,610 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:08,610 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,610 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:08,610 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,610 [198] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:52:08,610 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,610 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:52:08,610 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,610 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,610 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,610 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,610 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,610 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,610 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,611 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 4, 8, 610717), datetime.datetime(2023, 8, 8, 5, 52, 8, 610775)]) gunicorn-web stdout | 2023-08-08 05:52:08,612 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 05:52:08,612 [198] [DEBUG] [app] Ending request: urn:request:165652b5-f138-40f8-8fa3-18e73a2101fe (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:165652b5-f138-40f8-8fa3-18e73a2101fe', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:08,613 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.011 62 0.012) gunicorn-web stdout | 2023-08-08 05:52:08,613 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:52:08,616 [200] [DEBUG] [app] Ending request: urn:request:b2151f2e-d8cd-4cd9-bede-4f2be5348153 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:b2151f2e-d8cd-4cd9-bede-4f2be5348153', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:08,617 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 399 0.017) gunicorn-web stdout | 2023-08-08 05:52:08,617 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:52:08,698 [199] [DEBUG] [app] Starting request: urn:request:36d11ed9-faf0-47ef-b6f3-eec71afa7c46 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:08,698 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,698 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,699 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,705 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:08,706 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:08,706 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,706 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:08,706 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,706 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,706 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,706 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,709 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:08,715 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:08,716 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,719 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,723 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:08,724 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:08,727 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:52:08,839 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:52:08,840 [199] [DEBUG] [github.Requester] GET https://api.github.com/user {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:52:08 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"9cb1ef85201d07db343ca73c6767340ac0fc5fb90c826222a744bc0bb0ecfe2b"', 'last-modified': 'Fri, 04 Aug 2023 13:02:06 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4979', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '21', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '6858:0CBF:13C50C8:283230C:64D1D808'} {"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false,"name":"Hu Weihua","company":null,"blog":"","location":null,"email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":15,"public_gists":0,"followers":5,"following":0,"created_at":"2014-04-22T08:41:09Z","updated_at":"2023-08-04T13:02:06Z"} gunicorn-web stdout | 2023-08-08 05:52:08,905 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user/orgs HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:52:08,906 [199] [DEBUG] [github.Requester] GET https://api.github.com/user/orgs {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:52:08 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"d13b5efb65207c14dcee4455a3f1eaf0a71ec38da08cb11bc838e0fe4536cdf8"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:org, read:org, repo, user, write:org', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4978', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '22', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '6858:0CBF:13C50E4:2832345:64D1D808'} [{"login":"finley","id":422903,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQyMjkwMw==","url":"https://api.github.com/orgs/finley","repos_url":"https://api.github.com/orgs/finley/repos","events_url":"https://api.github.com/orgs/finley/events","hooks_url":"https://api.github.com/orgs/finley/hooks","issues_url":"https://api.github.com/orgs/finley/issues","members_url":"https://api.github.com/orgs/finley/members{/member}","public_members_url":"https://api.github.com/orgs/finley/public_members{/member}","avatar_url":"https://avatars.githubusercontent.com/u/422903?v=4","description":null},{"login":"xcat2","id":10124414,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwMTI0NDE0","url":"https://api.github.com/orgs/xcat2","repos_url":"https://api.github.com/orgs/xcat2/repos","events_url":"https://api.github.com/orgs/xcat2/events","hooks_url":"https://api.github.com/orgs/xcat2/hooks","issues_url":"https://api.github.com/orgs/xcat2/issues","members_url":"https://api.github.com/orgs/xcat2/members{/member}","public_members_url":"https://api.github.com/orgs/xcat2/public_members{/member}","avatar_url":"https://avatars.githubusercontent.com/u/10124414?v=4","description":" xCAT offers complete management for HPC clusters, RenderFarms, Grids, WebFarms, Online Gaming Infrastructure, Clouds, Datacenters"}] gunicorn-web stdout | 2023-08-08 05:52:08,909 [199] [DEBUG] [app] Ending request: urn:request:36d11ed9-faf0-47ef-b6f3-eec71afa7c46 (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces) {'endpoint': 'api.buildtriggersourcenamespaces', 'request_id': 'urn:request:36d11ed9-faf0-47ef-b6f3-eec71afa7c46', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:08,910 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:08,911 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces HTTP/1.0" 200 492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:08 +0000] "GET /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e/namespaces HTTP/2.0" 200 492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.215 101 0.215) gcworker stdout | 2023-08-08 05:52:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:52:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:19.417660+00:00 (in 9.999454 seconds) gcworker stdout | 2023-08-08 05:52:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:19 UTC)" (scheduled at 2023-08-08 05:52:09.417660+00:00) gcworker stdout | 2023-08-08 05:52:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264329418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:52:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:52:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:52:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:52:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:52:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:04.187429+00:00 (in 54.393243 seconds) repositorygcworker stdout | 2023-08-08 05:52:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:09 UTC)" (scheduled at 2023-08-08 05:52:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:52:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:52:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 9, 794858), True, datetime.datetime(2023, 8, 8, 5, 52, 9, 794858), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:52:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:52:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:52:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:09 UTC)" executed successfully builder stdout | 2023-08-08 05:52:10,097 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:52:10,106 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:52:10,106 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:52:10,110 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:52:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:52:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:22.293757+00:00 (in 9.999395 seconds) chunkcleanupworker stdout | 2023-08-08 05:52:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:22 UTC)" (scheduled at 2023-08-08 05:52:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:52:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:52:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 12, 294892), True, datetime.datetime(2023, 8, 8, 5, 52, 12, 294892), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:52:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:52:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:52:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:52:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:52:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:54.284093+00:00 (in 41.186190 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:52:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:13 UTC)" (scheduled at 2023-08-08 05:52:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:52:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:52:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 13, 98453), True, datetime.datetime(2023, 8, 8, 5, 52, 13, 98453), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:52:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:52:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:52:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:13 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:13,548 [198] [DEBUG] [app] Starting request: urn:request:66847121-eb9c-43f5-ae24-12f655b46c4d (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:13,548 [197] [DEBUG] [app] Starting request: urn:request:82220d8e-5b02-4896-97ce-ab2ebc7475b9 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:13,548 [200] [DEBUG] [app] Starting request: urn:request:ea8dc8ac-659a-4588-8ef7-720d1cbb3fca (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:13,548 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,548 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,548 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,548 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,548 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,548 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,549 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,549 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,549 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,555 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:13,555 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:13,555 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,556 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,556 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:13,556 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:13,556 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:13,556 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:52:13,556 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:13,556 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:13,557 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:13,557 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,557 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:13,557 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,557 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,557 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:13,557 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:13,557 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:13,557 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:13,557 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:52:13,558 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:52:13,558 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,558 [198] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'whuquay', 101]) gunicorn-web stdout | 2023-08-08 05:52:13,560 [200] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'build_test', 101]) gunicorn-web stdout | 2023-08-08 05:52:13,561 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:52:13,562 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:52:13,563 [200] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Max("t1"."lifetime_start_ms") FROM "tag" AS "t1" WHERE ((("t1"."repository_id" IN (%s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) GROUP BY "t1"."repository_id"', [2, None, 1691473933562, False]) gunicorn-web stdout | 2023-08-08 05:52:13,563 [197] [DEBUG] [app] Ending request: urn:request:82220d8e-5b02-4896-97ce-ab2ebc7475b9 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:82220d8e-5b02-4896-97ce-ab2ebc7475b9', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:13,564 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:13,564 [198] [DEBUG] [app] Ending request: urn:request:66847121-eb9c-43f5-ae24-12f655b46c4d (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:66847121-eb9c-43f5-ae24-12f655b46c4d', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'whuquay', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:13,564 [200] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Sum("t1"."count") FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" IN (%s)) AND ("t1"."date" >= %s)) GROUP BY "t1"."repository_id"', [2, datetime.date(2023, 8, 1)]) gunicorn-web stdout | 2023-08-08 05:52:13,564 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:13 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:13 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 156 0.018) gunicorn-web stdout | 2023-08-08 05:52:13,565 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:13,565 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:13 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:13 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.019 103 0.019) gunicorn-web stdout | 2023-08-08 05:52:13,566 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 05:52:13,569 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,571 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 05:52:13,572 [200] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,574 [200] [DEBUG] [peewee] ('SELECT Sum("t1"."layers_compressed_size") AS "size_bytes" FROM "manifest" AS "t1" INNER JOIN (SELECT "t2"."manifest_id" FROM "tag" AS "t2" WHERE ((("t2"."hidden" = %s) AND (("t2"."lifetime_end_ms" IS %s) OR ("t2"."lifetime_end_ms" > %s))) AND ("t2"."repository_id" = %s)) GROUP BY "t2"."manifest_id" HAVING (Count("t2"."name") > %s)) AS "t3" ON ("t3"."manifest_id" = "t1"."id") WHERE ("t1"."repository_id" = %s)', [False, None, 1691473933573, 2, 0, 2]) gunicorn-web stdout | 2023-08-08 05:52:13,575 [200] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,576 [200] [DEBUG] [peewee] ('UPDATE "repositorysize" SET "size_bytes" = %s WHERE ("repositorysize"."repository_id" = %s)', [0, 2]) gunicorn-web stdout | 2023-08-08 05:52:13,579 [200] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:13,581 [200] [DEBUG] [app] Ending request: urn:request:ea8dc8ac-659a-4588-8ef7-720d1cbb3fca (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:ea8dc8ac-659a-4588-8ef7-720d1cbb3fca', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'false', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:13,582 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:13 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/2.0" 200 274 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.036 114 0.036) gunicorn-web stdout | 2023-08-08 05:52:13,582 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:13 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/1.0" 200 274 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:52:15,309 [200] [DEBUG] [app] Starting request: urn:request:5dd493e4-48a1-4d70-973c-aeda34dba572 (/api/v1/repository/build_test/github/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:15,309 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,309 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,310 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:15,310 [198] [DEBUG] [app] Starting request: urn:request:d699dbd7-e44e-4927-a54c-081e453d0c5c (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:15,310 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,310 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,311 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:15,317 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:15,317 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:15,317 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,317 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:15,317 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,318 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:15,318 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:15,318 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:15,318 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:15,318 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:15,318 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,318 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:15,318 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,318 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:15,318 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:15,319 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:15,320 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:15,322 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:15,326 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:15,327 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:15,328 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:15,329 [198] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:15,329 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:15,330 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:15,331 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [2, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:15,332 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [2, None, 1691473935332, False, 101, 0]) gunicorn-web stdout | 2023-08-08 05:52:15,333 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [2, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:52:15,334 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,334 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:15,335 [200] [DEBUG] [app] Ending request: urn:request:5dd493e4-48a1-4d70-973c-aeda34dba572 (/api/v1/repository/build_test/github/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:5dd493e4-48a1-4d70-973c-aeda34dba572', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/github/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:15,335 [198] [DEBUG] [app] Ending request: urn:request:d699dbd7-e44e-4927-a54c-081e453d0c5c (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:d699dbd7-e44e-4927-a54c-081e453d0c5c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:15,335 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:15,336 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:15 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:15 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 520 0.028) gunicorn-web stdout | 2023-08-08 05:52:15,336 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:15,336 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:15 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:15 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/2.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 88 0.028) notificationworker stdout | 2023-08-08 05:52:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:52:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:25.371682+00:00 (in 9.999428 seconds) notificationworker stdout | 2023-08-08 05:52:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:25 UTC)" (scheduled at 2023-08-08 05:52:15.371682+00:00) notificationworker stdout | 2023-08-08 05:52:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:52:15,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 15, 372833), True, datetime.datetime(2023, 8, 8, 5, 52, 15, 372833), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:52:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:52:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:52:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:16,215 [199] [DEBUG] [app] Starting request: urn:request:9af53d58-f11a-4a94-a01b-77d7e7689d84 (/repository/build_test/github/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:16,217 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:16,224 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:16,225 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:16,229 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['github', 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:16,234 [199] [DEBUG] [app] Ending request: urn:request:9af53d58-f11a-4a94-a01b-77d7e7689d84 (/repository/build_test/github/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:9af53d58-f11a-4a94-a01b-77d7e7689d84', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f', 'path': '/repository/build_test/github/status', 'parameters': {'token': '245e22c3-ba4f-4e53-9a58-fce8fb4eb90f'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:16,235 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:16,236 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:16 +0000] "GET /repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:16 +0000] "GET /repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 434 0.022) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:16 +0000] "GET /static/fonts/glyphicons-halflings-regular.woff2 HTTP/2.0" 200 18028 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/static/ldn/bootstrap.min.css-13f668494557.css" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 355 -) gunicorn-web stdout | 2023-08-08 05:52:16,387 [199] [DEBUG] [app] Starting request: urn:request:4fc7782c-03c6-44e2-b1ad-91c60900b346 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:16,387 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:16,387 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:16,388 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:16,394 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:16,394 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:16,394 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:16,394 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:16,394 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:16,395 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:16,395 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:16,395 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:16,397 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:16,403 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:16,404 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:16,408 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:52:16,411 [199] [DEBUG] [app] Ending request: urn:request:4fc7782c-03c6-44e2-b1ad-91c60900b346 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:4fc7782c-03c6-44e2-b1ad-91c60900b346', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:16,412 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:16,412 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:16 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:16 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 132 0.027) notificationworker stdout | 2023-08-08 05:52:17,034 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gunicorn-web stdout | 2023-08-08 05:52:17,721 [197] [DEBUG] [app] Starting request: urn:request:c8df35ca-de70-43bf-96ba-aea2060a4024 (/api/v1/repository/build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:17,721 [200] [DEBUG] [app] Starting request: urn:request:0d24a62c-1509-4e8b-9601-d509b633ce0c (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:17,721 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:17,722 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:17,722 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:17,722 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:17,722 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:17,722 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:17,729 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:17,729 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:17,729 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:17,729 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:17,729 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:17,729 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:17,729 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:17,729 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:17,729 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:17,729 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:17,729 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:17,729 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:17,729 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:17,729 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:17,729 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:17,729 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:17,732 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:17,732 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:17,738 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:17,738 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:17,739 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:17,740 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:17,742 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'github']) gunicorn-web stdout | 2023-08-08 05:52:17,743 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 10]) gunicorn-web stdout | 2023-08-08 05:52:17,745 [197] [DEBUG] [app] Ending request: urn:request:c8df35ca-de70-43bf-96ba-aea2060a4024 (/api/v1/repository/build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:c8df35ca-de70-43bf-96ba-aea2060a4024', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/', 'path': '/api/v1/repository/build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:17,746 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:17 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/2.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 535 0.026) gunicorn-web stdout | 2023-08-08 05:52:17,746 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:17 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/1.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:52:17,747 [200] [DEBUG] [app] Ending request: urn:request:0d24a62c-1509-4e8b-9601-d509b633ce0c (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:0d24a62c-1509-4e8b-9601-d509b633ce0c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=10', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:17,748 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:17 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 73 0.028) gunicorn-web stdout | 2023-08-08 05:52:17,748 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:17 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" logrotateworker stdout | 2023-08-08 05:52:18,214 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:52:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:52:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:29.417660+00:00 (in 9.999484 seconds) gcworker stdout | 2023-08-08 05:52:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:29 UTC)" (scheduled at 2023-08-08 05:52:19.417660+00:00) gcworker stdout | 2023-08-08 05:52:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264339418, 1209600, True, 3, 500, 1, 0]) manifestbackfillworker stdout | 2023-08-08 05:52:19,420 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gcworker stdout | 2023-08-08 05:52:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:52:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:52:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:29 UTC)" executed successfully securityworker stdout | 2023-08-08 05:52:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:52:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:26.890741+00:00 (in 7.202806 seconds) securityworker stdout | 2023-08-08 05:52:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:49 UTC)" (scheduled at 2023-08-08 05:52:19.687363+00:00) securityworker stdout | 2023-08-08 05:52:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:52:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:52:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:52:19,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:52:19,700 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:52:19,701 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:52:19,701 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:49 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:52:19,844 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:52:20,119 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:52:20,128 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:52:20,128 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:52:20,132 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:22,181 [199] [DEBUG] [app] Starting request: urn:request:310cc109-5c2b-4979-bc2a-da2831802c46 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:52:22,181 [197] [DEBUG] [app] Starting request: urn:request:ef52e4f2-014d-45cd-9a5e-1bc35d200e04 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:52:22,183 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:22,183 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:22,186 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:22,188 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:52:22,188 [209] [DEBUG] [app] Starting request: urn:request:9195889b-a765-4cd2-af9e-8efeeb1f98e5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:52:22,189 [210] [DEBUG] [app] Starting request: urn:request:a022c451-ea3f-4c53-b781-e7405adc973e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:52:22,189 [209] [DEBUG] [app] Ending request: urn:request:9195889b-a765-4cd2-af9e-8efeeb1f98e5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9195889b-a765-4cd2-af9e-8efeeb1f98e5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:52:22,189 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:52:22,189 [210] [DEBUG] [app] Ending request: urn:request:a022c451-ea3f-4c53-b781-e7405adc973e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a022c451-ea3f-4c53-b781-e7405adc973e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:52:22,190 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:52:22,190 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:52:22,190 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:22,192 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:22,192 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:22,195 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:22,196 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:22,196 [199] [DEBUG] [app] Starting request: urn:request:e6c42829-77d9-40f2-98eb-ff59ce554d37 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:52:22,196 [199] [DEBUG] [app] Ending request: urn:request:e6c42829-77d9-40f2-98eb-ff59ce554d37 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e6c42829-77d9-40f2-98eb-ff59ce554d37', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:52:22,197 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:52:22,197 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:22,197 [198] [DEBUG] [app] Starting request: urn:request:189ce91e-6f17-48e8-bd2c-169e14728387 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:52:22,197 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:52:22,197 [198] [DEBUG] [app] Ending request: urn:request:189ce91e-6f17-48e8-bd2c-169e14728387 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:189ce91e-6f17-48e8-bd2c-169e14728387', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:52:22,198 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:52:22,198 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:22,199 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:52:22,203 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:52:22,204 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:52:22,208 [199] [DEBUG] [app] Ending request: urn:request:310cc109-5c2b-4979-bc2a-da2831802c46 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:310cc109-5c2b-4979-bc2a-da2831802c46', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:52:22,208 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:22,209 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:52:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:52:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.029) gunicorn-web stdout | 2023-08-08 05:52:22,209 [197] [DEBUG] [app] Ending request: urn:request:ef52e4f2-014d-45cd-9a5e-1bc35d200e04 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ef52e4f2-014d-45cd-9a5e-1bc35d200e04', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:52:22,209 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:22,210 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:52:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:52:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.029) chunkcleanupworker stdout | 2023-08-08 05:52:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:52:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:28.874906+00:00 (in 6.580546 seconds) chunkcleanupworker stdout | 2023-08-08 05:52:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:32 UTC)" (scheduled at 2023-08-08 05:52:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:52:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:52:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 22, 294955), True, datetime.datetime(2023, 8, 8, 5, 52, 22, 294955), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:52:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:52:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:52:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:52:22,629 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:52:24,434 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:52:24,486 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:52:24,708 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:52:24,710 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:52:24,714 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:52:24,750 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:52:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:52:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:35.371682+00:00 (in 9.999391 seconds) notificationworker stdout | 2023-08-08 05:52:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:35 UTC)" (scheduled at 2023-08-08 05:52:25.371682+00:00) notificationworker stdout | 2023-08-08 05:52:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:52:25,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 25, 372839), True, datetime.datetime(2023, 8, 8, 5, 52, 25, 372839), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:52:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:52:25,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:52:25,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:52:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:52:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:47.398551+00:00 (in 21.883668 seconds) exportactionlogsworker stdout | 2023-08-08 05:52:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:25 UTC)" (scheduled at 2023-08-08 05:52:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:52:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:52:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:52:25,886 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:52:26,441 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:52:26,643 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:52:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:52:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:49.687363+00:00 (in 22.796005 seconds) securityworker stdout | 2023-08-08 05:52:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:56 UTC)" (scheduled at 2023-08-08 05:52:26.890741+00:00) securityworker stdout | 2023-08-08 05:52:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:52:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:52:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:52:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:52:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:52:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:52:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:52:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:52:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:52:26,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:52:26,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:52:26,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:52:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:52:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:52:26,904 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:52:26,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:52:26,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:52:26,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:52:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:27,109 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:52:27,115 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:52:27,118 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:52:27,124 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:52:27,128 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:52:27,840 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:52:27,922 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:52:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:52:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:32.293757+00:00 (in 3.418217 seconds) chunkcleanupworker stdout | 2023-08-08 05:52:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:28 UTC)" (scheduled at 2023-08-08 05:52:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:52:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:52:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:52:29,046 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:52:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:52:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:39.417660+00:00 (in 9.999511 seconds) gcworker stdout | 2023-08-08 05:52:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:39 UTC)" (scheduled at 2023-08-08 05:52:29.417660+00:00) gcworker stdout | 2023-08-08 05:52:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264349418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:52:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:52:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:52:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:52:29,874 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:52:29,963 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:52:30,136 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:52:30,144 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:52:30,145 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:52:30,149 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:52:30,449 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:52:31,017 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:52:31,316 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:52:31,575 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:52:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:52:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:42.293757+00:00 (in 9.999441 seconds) chunkcleanupworker stdout | 2023-08-08 05:52:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:42 UTC)" (scheduled at 2023-08-08 05:52:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:52:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:52:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 32, 294810), True, datetime.datetime(2023, 8, 8, 5, 52, 32, 294810), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:52:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:52:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:52:32,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:42 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:32,835 [200] [DEBUG] [app] Starting request: urn:request:1628aaa1-d874-4b32-bfdf-176d0c64dcae (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:32,835 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:32,835 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:32,836 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:32,842 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:32,842 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:32,843 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:32,843 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:32,843 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:32,843 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:32,843 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:32,843 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:32,845 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:32,852 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:32,853 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:32,856 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:32,858 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:32,862 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-registry stdout | 2023-08-08 05:52:32,914 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:52:32,917 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:52:32,920 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:52:32,924 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:52:32,928 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:52:32,931 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:52:32,933 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:52:32,936 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:52:32,939 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-web stdout | 2023-08-08 05:52:32,984 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:52:32,984 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:52:32 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4977', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '23', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E230:7F9F:11FD9AB:249A589:64D1D820'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAREJZXCMZCBUAVOM3TE2HMUY","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:52:33,063 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/keys/85930047 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:52:33,064 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/keys/85930047 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:52:33 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"a065b44f90f0748d6f25ace608b6f11a5ba7219226665f757bd2fe253440ae23"', 'last-modified': 'Tue, 08 Aug 2023 05:51:21 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4976', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '24', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E230:7F9F:11FD9D2:249A5D6:64D1D820'} {"id":85930047,"key":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnLN7FCYWGm7C1durrXhAAKp6NnijqiYjLrPNhw11ohhKjX/DH+qBDJyJBirBGcpMsjRqfxWiVow1fWd6Z2DMa0gruZuhO3c4orINe+LSyCb+RgvEc7UXBSJzhWGeOWR+5Xu3mOOWrc3rESfpV53z8Wo765pdiMoSQnsrTHVRm32pT5FcePiRelkoJTq1VfzCVdI35KEKBimfsVjmG+maqtL0obUQEejbhsJTO656p1zwklUQqKhsHGBbFk+5S8I1hbmAzdnEB4L4UYoGAU1VxrANCU3wRD186xtdAYECNRENp9k4gi0Mz/YiyhH58sfSWh+Nr6MKdYVHxob5tviWZ","url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys/85930047","title":"Red Hat Quay Builder","verified":true,"created_at":"2023-08-08T05:51:21Z","read_only":false,"last_used":null,"added_by":"hu-weihua"} gunicorn-web stdout | 2023-08-08 05:52:33,202 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "DELETE /repos/hu-weihua/quay_build_test/keys/85930047 HTTP/1.1" 204 0 gunicorn-web stdout | 2023-08-08 05:52:33,202 [200] [DEBUG] [github.Requester] DELETE https://api.github.com/repos/hu-weihua/quay_build_test/keys/85930047 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 204 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:52:33 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4975', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '25', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'vary': 'Accept-Encoding, Accept, X-Requested-With', 'x-github-request-id': 'E230:7F9F:11FD9F0:249A617:64D1D821'} gunicorn-web stdout | 2023-08-08 05:52:33,303 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/hooks/427817209 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:52:33,303 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:52:33 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"a04cd48e5303dcee21317b180ff5dc776d566e979d7624fba65fb6860a47c690"', 'last-modified': 'Tue, 08 Aug 2023 05:51:21 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:repo_hook, read:repo_hook, repo, write:repo_hook', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4974', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '26', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E230:7F9F:11FDA25:249A687:64D1D821'} {"type":"Repository","id":427817209,"name":"web","active":true,"events":["push"],"config":{"content_type":"json","insecure_ssl":"0","url":"https://$token:3NVXDUTKEMB92FJZTI3ATYFY6ZRV9V630RPRD209U09WFR32PEBACK4QHSJ1HEQM@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e"},"updated_at":"2023-08-08T05:51:21Z","created_at":"2023-08-08T05:51:21Z","url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209","test_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209/test","ping_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209/pings","deliveries_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209/deliveries","last_response":{"code":502,"status":"connection_error","message":"failed to connect to host"}} gunicorn-web stdout | 2023-08-08 05:52:33,429 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "DELETE /repos/hu-weihua/quay_build_test/hooks/427817209 HTTP/1.1" 204 0 gunicorn-web stdout | 2023-08-08 05:52:33,430 [200] [DEBUG] [github.Requester] DELETE https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817209 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 204 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:52:33 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:repo_hook, repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4973', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '27', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'vary': 'Accept-Encoding, Accept, X-Requested-With', 'x-github-request-id': 'E230:7F9F:11FDA4A:249A6F4:64D1D821'} gunicorn-web stdout | 2023-08-08 05:52:33,432 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:33,435 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:33,436 [200] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [23, 2, 1, 2, datetime.datetime(2023, 8, 8, 5, 52, 33, 434930), '10.130.2.1', '{"repo": "github", "trigger_id": "3f8c84f2-7889-4e84-8777-d2fe1e46ea1e", "service": "github"}']) gunicorn-web stdout | 2023-08-08 05:52:33,440 [200] [DEBUG] [peewee] ('UPDATE "repositorybuild" SET "trigger_id" = %s WHERE ("repositorybuild"."trigger_id" = %s)', [None, 1]) gunicorn-web stdout | 2023-08-08 05:52:33,441 [200] [DEBUG] [peewee] ('DELETE FROM "repositorybuildtrigger" WHERE ("repositorybuildtrigger"."id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:52:33,444 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."friendly_name", "t1"."token_name", "t1"."token_code", "t1"."repository_id", "t1"."created", "t1"."role_id", "t1"."temporary", "t1"."kind_id" FROM "accesstoken" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:33,445 [200] [DEBUG] [peewee] ('DELETE FROM "accesstoken" WHERE ("accesstoken"."id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:52:33,449 [200] [DEBUG] [app] Ending request: urn:request:1628aaa1-d874-4b32-bfdf-176d0c64dcae (/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:1628aaa1-d874-4b32-bfdf-176d0c64dcae', 'remote_addr': '10.130.2.1', 'http_method': 'DELETE', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'path': '/api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e', 'parameters': {}, 'json_body': {}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:33,450 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:33,451 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:33 +0000] "DELETE /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/1.0" 204 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:33 +0000] "DELETE /api/v1/repository/build_test/github/trigger/3f8c84f2-7889-4e84-8777-d2fe1e46ea1e HTTP/2.0" 204 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.618 990 0.618) notificationworker stdout | 2023-08-08 05:52:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:52:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:45.371682+00:00 (in 9.999500 seconds) notificationworker stdout | 2023-08-08 05:52:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:45 UTC)" (scheduled at 2023-08-08 05:52:35.371682+00:00) notificationworker stdout | 2023-08-08 05:52:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:52:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 35, 372695), True, datetime.datetime(2023, 8, 8, 5, 52, 35, 372695), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:52:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:52:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:52:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:52:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:52:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:54.393627+00:00 (in 17.813249 seconds) namespacegcworker stdout | 2023-08-08 05:52:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:36 UTC)" (scheduled at 2023-08-08 05:52:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:52:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:52:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:37,178 [197] [DEBUG] [app] Starting request: urn:request:6ff3d311-8370-4643-bc6a-73e755f2140a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:52:37,179 [200] [DEBUG] [app] Starting request: urn:request:5420d2c1-f167-43c3-81eb-cd8df735ac57 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:52:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:52:37,184 [209] [DEBUG] [app] Starting request: urn:request:3bfd7bdf-2aaf-4a16-9e24-e7246b6d46ab (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:52:37,185 [209] [DEBUG] [app] Ending request: urn:request:3bfd7bdf-2aaf-4a16-9e24-e7246b6d46ab (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3bfd7bdf-2aaf-4a16-9e24-e7246b6d46ab', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:52:37,185 [208] [DEBUG] [app] Starting request: urn:request:4db314d4-8c6f-4f7b-9a20-4700008b9894 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:52:37,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-registry stdout | 2023-08-08 05:52:37,185 [208] [DEBUG] [app] Ending request: urn:request:4db314d4-8c6f-4f7b-9a20-4700008b9894 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4db314d4-8c6f-4f7b-9a20-4700008b9894', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:52:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:52:37,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:52:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:37,190 [199] [DEBUG] [app] Starting request: urn:request:908f053e-531c-410b-90b3-0cc82bd33c05 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:52:37,191 [197] [DEBUG] [app] Starting request: urn:request:38c2c011-6240-41ee-8382-c95b6d2495a3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:52:37,191 [199] [DEBUG] [app] Ending request: urn:request:908f053e-531c-410b-90b3-0cc82bd33c05 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:908f053e-531c-410b-90b3-0cc82bd33c05', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:52:37,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:52:37,191 [197] [DEBUG] [app] Ending request: urn:request:38c2c011-6240-41ee-8382-c95b6d2495a3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:38c2c011-6240-41ee-8382-c95b6d2495a3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:52:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:52:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:52:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:52:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:52:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 05:52:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:52:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:07.197412+00:00 (in 29.999527 seconds) buildlogsarchiver stdout | 2023-08-08 05:52:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:07 UTC)" (scheduled at 2023-08-08 05:52:37.197412+00:00) gunicorn-web stdout | 2023-08-08 05:52:37,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:52:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 52, 37, 198435), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:37,204 [197] [DEBUG] [app] Ending request: urn:request:6ff3d311-8370-4643-bc6a-73e755f2140a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6ff3d311-8370-4643-bc6a-73e755f2140a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:52:37,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:52:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:52:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) buildlogsarchiver stdout | 2023-08-08 05:52:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:52:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:52:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:37,207 [200] [DEBUG] [app] Ending request: urn:request:5420d2c1-f167-43c3-81eb-cd8df735ac57 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5420d2c1-f167-43c3-81eb-cd8df735ac57', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:52:37,207 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:37,208 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:52:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:52:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.030 47 0.030) gcworker stdout | 2023-08-08 05:52:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:52:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:49.417660+00:00 (in 9.999484 seconds) gcworker stdout | 2023-08-08 05:52:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:49 UTC)" (scheduled at 2023-08-08 05:52:39.417660+00:00) gcworker stdout | 2023-08-08 05:52:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264359418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:52:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:52:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:52:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:49 UTC)" executed successfully builder stdout | 2023-08-08 05:52:40,160 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:52:40,169 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:52:40,169 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:52:40,173 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:52:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:52:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:52.293757+00:00 (in 9.999497 seconds) chunkcleanupworker stdout | 2023-08-08 05:52:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:52 UTC)" (scheduled at 2023-08-08 05:52:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:52:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:52:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 42, 294773), True, datetime.datetime(2023, 8, 8, 5, 52, 42, 294773), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:52:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:52:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:52:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:52:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:52:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:55.371682+00:00 (in 9.999480 seconds) notificationworker stdout | 2023-08-08 05:52:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:55 UTC)" (scheduled at 2023-08-08 05:52:45.371682+00:00) notificationworker stdout | 2023-08-08 05:52:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:52:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 45, 373208), True, datetime.datetime(2023, 8, 8, 5, 52, 45, 373208), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:52:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:52:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:52:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:52:47,069 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:52:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:52:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:25.514341+00:00 (in 38.115262 seconds) exportactionlogsworker stdout | 2023-08-08 05:52:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:47 UTC)" (scheduled at 2023-08-08 05:52:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:52:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:52:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 47, 399494), True, datetime.datetime(2023, 8, 8, 5, 52, 47, 399494), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:52:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:52:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:52:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:47 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:47,547 [197] [DEBUG] [app] Starting request: urn:request:17cb82f4-af2f-482b-9a57-cfdf65b35bc4 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:52:47,547 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:47,547 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:47,548 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:47,555 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:52:47,555 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:52:47,555 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:47,555 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:52:47,555 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:52:47,555 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:47,555 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:47,555 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:52:47,557 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:52:47,564 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:52:47,565 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:52:47,569 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:52:47,573 [197] [DEBUG] [app] Ending request: urn:request:17cb82f4-af2f-482b-9a57-cfdf65b35bc4 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:17cb82f4-af2f-482b-9a57-cfdf65b35bc4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:52:47,573 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:47,574 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:52:47 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:52:47 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 867 0.028) logrotateworker stdout | 2023-08-08 05:52:48,250 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:52:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:52:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:59.417660+00:00 (in 9.999478 seconds) gcworker stdout | 2023-08-08 05:52:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:59 UTC)" (scheduled at 2023-08-08 05:52:49.417660+00:00) gcworker stdout | 2023-08-08 05:52:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264369418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:52:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:52:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:52:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:52:59 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:52:49,449 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} securityworker stdout | 2023-08-08 05:52:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:52:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:52:56.890741+00:00 (in 7.202917 seconds) securityworker stdout | 2023-08-08 05:52:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:19 UTC)" (scheduled at 2023-08-08 05:52:49.687363+00:00) securityworker stdout | 2023-08-08 05:52:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:52:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:52:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:52:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:52:49,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:52:49,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:52:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:19 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:52:49,851 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:52:50,183 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:52:50,192 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:52:50,192 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:52:50,197 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:52,179 [198] [DEBUG] [app] Starting request: urn:request:611ff4de-b3ff-4048-a6ba-aa65db573029 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:52:52,179 [200] [DEBUG] [app] Starting request: urn:request:95ff92bf-a0cb-45f0-a405-31414ebacd17 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:52:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:52:52,185 [203] [DEBUG] [app] Starting request: urn:request:18fe78da-d100-43ba-b913-5f99c1601cf1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:52:52,185 [207] [DEBUG] [app] Starting request: urn:request:86a33332-500f-4dbd-8e25-43fd4e9a5a69 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:52:52,185 [203] [DEBUG] [app] Ending request: urn:request:18fe78da-d100-43ba-b913-5f99c1601cf1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:18fe78da-d100-43ba-b913-5f99c1601cf1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:52:52,186 [207] [DEBUG] [app] Ending request: urn:request:86a33332-500f-4dbd-8e25-43fd4e9a5a69 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:86a33332-500f-4dbd-8e25-43fd4e9a5a69', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:52:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:52:52,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:52:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 05:52:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:52:52,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:52,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:52:52,191 [198] [DEBUG] [app] Starting request: urn:request:ae1cb5c0-d8d8-4976-bbf0-ceaee721992b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:52:52,191 [198] [DEBUG] [app] Ending request: urn:request:ae1cb5c0-d8d8-4976-bbf0-ceaee721992b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ae1cb5c0-d8d8-4976-bbf0-ceaee721992b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:52:52,192 [197] [DEBUG] [app] Starting request: urn:request:46b6b123-d83a-4806-8fee-d302de972125 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:52:52,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:52:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:52,192 [197] [DEBUG] [app] Ending request: urn:request:46b6b123-d83a-4806-8fee-d302de972125 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:46b6b123-d83a-4806-8fee-d302de972125', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:52:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:52:52,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:52:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:52:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:52:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:52:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:52:52,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:52:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:52:52,203 [198] [DEBUG] [app] Ending request: urn:request:611ff4de-b3ff-4048-a6ba-aa65db573029 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:611ff4de-b3ff-4048-a6ba-aa65db573029', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:52:52,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:52:52,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:52:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:52:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:52:52,204 [200] [DEBUG] [app] Ending request: urn:request:95ff92bf-a0cb-45f0-a405-31414ebacd17 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:95ff92bf-a0cb-45f0-a405-31414ebacd17', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:52:52,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:52:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:52:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:52:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:52:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:52:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:02.293757+00:00 (in 9.999475 seconds) chunkcleanupworker stdout | 2023-08-08 05:52:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:02 UTC)" (scheduled at 2023-08-08 05:52:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:52:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:52:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 52, 294860), True, datetime.datetime(2023, 8, 8, 5, 52, 52, 294860), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:52:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:52:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:52:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:52:52,666 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:52:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:52:54,285 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:13.097375+00:00 (in 18.812464 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:52:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:54 UTC)" (scheduled at 2023-08-08 05:52:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:52:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:52:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:52:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:52:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:36.579841+00:00 (in 42.185597 seconds) namespacegcworker stdout | 2023-08-08 05:52:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:54 UTC)" (scheduled at 2023-08-08 05:52:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:52:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:52:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 54, 394806), True, datetime.datetime(2023, 8, 8, 5, 52, 54, 394806), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:52:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:52:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:52:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:53:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:52:54,455 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:52:54,522 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:52:54,717 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:52:54,720 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:52:54,722 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:52:54,779 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:52:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:52:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:02.178530+00:00 (in 6.806362 seconds) notificationworker stdout | 2023-08-08 05:52:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:05 UTC)" (scheduled at 2023-08-08 05:52:55.371682+00:00) notificationworker stdout | 2023-08-08 05:52:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:52:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 52, 55, 373054), True, datetime.datetime(2023, 8, 8, 5, 52, 55, 373054), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:52:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:52:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:52:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:52:55,921 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:52:56,449 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:52:56,681 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:52:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:52:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:19.687363+00:00 (in 22.796048 seconds) securityworker stdout | 2023-08-08 05:52:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:26 UTC)" (scheduled at 2023-08-08 05:52:56.890741+00:00) securityworker stdout | 2023-08-08 05:52:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:52:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:52:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:52:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:52:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:52:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:52:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:52:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:52:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:52:56,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:52:56,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:52:56,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:52:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:52:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:52:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:52:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:52:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:52:56,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:52:57,118 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:52:57,127 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:52:57,130 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:52:57,137 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:52:57,140 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:52:57,877 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:52:57,956 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:52:59,071 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:52:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:52:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:09.417660+00:00 (in 9.999495 seconds) gcworker stdout | 2023-08-08 05:52:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:09 UTC)" (scheduled at 2023-08-08 05:52:59.417660+00:00) gcworker stdout | 2023-08-08 05:52:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264379418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:52:59,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:52:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:52:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:52:59,888 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:53:00,000 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:53:00,207 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:53:00,216 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:53:00,216 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:53:00,221 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:53:00,486 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:53:01,054 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:53:01,336 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:53:01,614 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:53:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:53:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:05.371682+00:00 (in 3.192659 seconds) notificationworker stdout | 2023-08-08 05:53:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:02 UTC)" (scheduled at 2023-08-08 05:53:02.178530+00:00) notificationworker stdout | 2023-08-08 05:53:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:53:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:53:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:53:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:12.293757+00:00 (in 9.999458 seconds) chunkcleanupworker stdout | 2023-08-08 05:53:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:12 UTC)" (scheduled at 2023-08-08 05:53:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:53:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:53:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 2, 294987), True, datetime.datetime(2023, 8, 8, 5, 53, 2, 294987), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:53:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:53:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:53:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:53:02,924 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:53:02,927 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:53:02,933 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:53:02,936 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:53:02,939 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:53:02,941 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:53:02,944 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:53:02,947 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:53:02,950 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:53:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:53:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:09.793658+00:00 (in 5.605655 seconds) repositorygcworker stdout | 2023-08-08 05:53:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:04 UTC)" (scheduled at 2023-08-08 05:53:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:53:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:53:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:53:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:53:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:15.371682+00:00 (in 9.999483 seconds) notificationworker stdout | 2023-08-08 05:53:05,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:15 UTC)" (scheduled at 2023-08-08 05:53:05.371682+00:00) notificationworker stdout | 2023-08-08 05:53:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:53:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 5, 373262), True, datetime.datetime(2023, 8, 8, 5, 53, 5, 373262), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:53:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:53:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:53:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:53:07,179 [197] [DEBUG] [app] Starting request: urn:request:0e3f1e1d-e926-46c7-bcd6-46f7ef80f7bd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:53:07,180 [200] [DEBUG] [app] Starting request: urn:request:86ee1440-b241-4bf3-9f11-daa83f27075b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:53:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:07,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:07,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:53:07,186 [203] [DEBUG] [app] Starting request: urn:request:8774b8e6-5304-47c6-8e21-16afee0966e0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:53:07,186 [209] [DEBUG] [app] Starting request: urn:request:80f0ecd5-984a-4ec6-82a3-cae57d8dce13 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:53:07,186 [203] [DEBUG] [app] Ending request: urn:request:8774b8e6-5304-47c6-8e21-16afee0966e0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8774b8e6-5304-47c6-8e21-16afee0966e0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:53:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:53:07,187 [209] [DEBUG] [app] Ending request: urn:request:80f0ecd5-984a-4ec6-82a3-cae57d8dce13 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:80f0ecd5-984a-4ec6-82a3-cae57d8dce13', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:53:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:53:07,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:53:07,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:07,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:07,191 [199] [DEBUG] [app] Starting request: urn:request:10ae0d35-f79d-469d-91c2-574bf28e1564 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:53:07,192 [197] [DEBUG] [app] Starting request: urn:request:5b9fb225-cd62-4628-a0e7-a71f522c431c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:53:07,192 [199] [DEBUG] [app] Ending request: urn:request:10ae0d35-f79d-469d-91c2-574bf28e1564 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:10ae0d35-f79d-469d-91c2-574bf28e1564', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:53:07,192 [197] [DEBUG] [app] Ending request: urn:request:5b9fb225-cd62-4628-a0e7-a71f522c431c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5b9fb225-cd62-4628-a0e7-a71f522c431c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:53:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:53:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:53:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:53:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:53:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:53:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:37.197412+00:00 (in 29.999512 seconds) buildlogsarchiver stdout | 2023-08-08 05:53:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:37 UTC)" (scheduled at 2023-08-08 05:53:07.197412+00:00) gunicorn-web stdout | 2023-08-08 05:53:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:53:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 53, 7, 198366), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:53:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:53:07,203 [197] [DEBUG] [app] Ending request: urn:request:0e3f1e1d-e926-46c7-bcd6-46f7ef80f7bd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0e3f1e1d-e926-46c7-bcd6-46f7ef80f7bd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:53:07,204 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:53:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:53:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:53:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:53:07,204 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:53:07,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:53:07,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:37 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:53:07,205 [200] [DEBUG] [app] Ending request: urn:request:86ee1440-b241-4bf3-9f11-daa83f27075b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:86ee1440-b241-4bf3-9f11-daa83f27075b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:53:07,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:53:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 05:53:07,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:53:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gcworker stdout | 2023-08-08 05:53:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:53:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:19.417660+00:00 (in 9.999465 seconds) gcworker stdout | 2023-08-08 05:53:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:19 UTC)" (scheduled at 2023-08-08 05:53:09.417660+00:00) gcworker stdout | 2023-08-08 05:53:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264389418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:53:09,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:53:09,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:53:09,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:53:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:53:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:04.187429+00:00 (in 54.393185 seconds) repositorygcworker stdout | 2023-08-08 05:53:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:09 UTC)" (scheduled at 2023-08-08 05:53:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:53:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:53:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 9, 794846), True, datetime.datetime(2023, 8, 8, 5, 53, 9, 794846), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:53:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:53:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:53:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:09 UTC)" executed successfully builder stdout | 2023-08-08 05:53:10,231 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:53:10,240 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:53:10,240 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:53:10,244 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:53:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:53:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:22.293757+00:00 (in 9.999423 seconds) chunkcleanupworker stdout | 2023-08-08 05:53:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:22 UTC)" (scheduled at 2023-08-08 05:53:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:53:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:53:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 12, 294863), True, datetime.datetime(2023, 8, 8, 5, 53, 12, 294863), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:53:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:53:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:53:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:53:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:53:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:54.284093+00:00 (in 41.186184 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:53:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:13 UTC)" (scheduled at 2023-08-08 05:53:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:53:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:53:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 13, 98261), True, datetime.datetime(2023, 8, 8, 5, 53, 13, 98261), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:53:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:53:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:53:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:53:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:53:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:25.371682+00:00 (in 9.999481 seconds) notificationworker stdout | 2023-08-08 05:53:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:25 UTC)" (scheduled at 2023-08-08 05:53:15.371682+00:00) notificationworker stdout | 2023-08-08 05:53:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:53:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 15, 372783), True, datetime.datetime(2023, 8, 8, 5, 53, 15, 372783), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:53:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:53:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:53:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:53:17,107 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:53:18,290 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 05:53:18,806 [199] [DEBUG] [app] Starting request: urn:request:9e3bd27a-0beb-493f-ac25-4e485d787fbd (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:53:18,806 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:53:18,806 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:53:18,807 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:53:18,813 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:53:18,814 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:53:18,814 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:53:18,814 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:53:18,814 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:53:18,814 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:53:18,814 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:53:18,814 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:53:18,816 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:53:18,823 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:53:18,824 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:53:18,828 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:53:18,832 [199] [DEBUG] [app] Ending request: urn:request:9e3bd27a-0beb-493f-ac25-4e485d787fbd (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:9e3bd27a-0beb-493f-ac25-4e485d787fbd', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:53:18,833 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:18,833 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:53:18 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:53:18 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 867 0.028) gcworker stdout | 2023-08-08 05:53:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:53:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:29.417660+00:00 (in 9.999461 seconds) gcworker stdout | 2023-08-08 05:53:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:29 UTC)" (scheduled at 2023-08-08 05:53:19.417660+00:00) gcworker stdout | 2023-08-08 05:53:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264399418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:53:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:53:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:53:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:29 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:53:19,465 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} securityworker stdout | 2023-08-08 05:53:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:53:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:26.890741+00:00 (in 7.202910 seconds) securityworker stdout | 2023-08-08 05:53:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:49 UTC)" (scheduled at 2023-08-08 05:53:19.687363+00:00) securityworker stdout | 2023-08-08 05:53:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:53:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:53:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:53:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:53:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:53:19,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:53:19,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:49 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:53:19,877 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:53:20,255 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:53:20,264 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:53:20,264 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:53:20,268 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:22,179 [200] [DEBUG] [app] Starting request: urn:request:2ae3b19c-7bf2-4351-ab3f-abb486dea877 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:53:22,180 [197] [DEBUG] [app] Starting request: urn:request:9def60b8-ebb5-429b-b7e1-d6aa6072fbbc (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:53:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:22,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:22,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:53:22,185 [203] [DEBUG] [app] Starting request: urn:request:590f10aa-fc6b-41da-bb61-031d37c134ed (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:53:22,186 [203] [DEBUG] [app] Ending request: urn:request:590f10aa-fc6b-41da-bb61-031d37c134ed (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:590f10aa-fc6b-41da-bb61-031d37c134ed', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:53:22,186 [210] [DEBUG] [app] Starting request: urn:request:d14aa9fb-175f-48e4-a72a-3c083411e982 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:53:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:53:22,186 [210] [DEBUG] [app] Ending request: urn:request:d14aa9fb-175f-48e4-a72a-3c083411e982 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d14aa9fb-175f-48e4-a72a-3c083411e982', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:53:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:53:22,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:53:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:22,191 [198] [DEBUG] [app] Starting request: urn:request:ff18599e-14cb-4fb5-b211-135dd660751c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:53:22,192 [198] [DEBUG] [app] Ending request: urn:request:ff18599e-14cb-4fb5-b211-135dd660751c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ff18599e-14cb-4fb5-b211-135dd660751c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:53:22,192 [200] [DEBUG] [app] Starting request: urn:request:70102fd1-b8c8-462e-b426-7c6dca2be2b1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:53:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:53:22,192 [200] [DEBUG] [app] Ending request: urn:request:70102fd1-b8c8-462e-b426-7c6dca2be2b1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:70102fd1-b8c8-462e-b426-7c6dca2be2b1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:53:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:53:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:53:22,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:53:22,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:53:22,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:53:22,204 [197] [DEBUG] [app] Ending request: urn:request:9def60b8-ebb5-429b-b7e1-d6aa6072fbbc (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9def60b8-ebb5-429b-b7e1-d6aa6072fbbc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:53:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:22,204 [200] [DEBUG] [app] Ending request: urn:request:2ae3b19c-7bf2-4351-ab3f-abb486dea877 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2ae3b19c-7bf2-4351-ab3f-abb486dea877', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:53:22,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:53:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:53:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:53:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:53:22,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:53:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:53:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:53:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:28.874906+00:00 (in 6.580567 seconds) chunkcleanupworker stdout | 2023-08-08 05:53:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:32 UTC)" (scheduled at 2023-08-08 05:53:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:53:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:53:22,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 22, 295017), True, datetime.datetime(2023, 8, 8, 5, 53, 22, 295017), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:53:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:53:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:53:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:53:22,702 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:53:24,489 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:53:24,545 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:53:24,725 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:53:24,728 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:53:24,731 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:53:24,815 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:53:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:53:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:35.371682+00:00 (in 9.999465 seconds) notificationworker stdout | 2023-08-08 05:53:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:35 UTC)" (scheduled at 2023-08-08 05:53:25.371682+00:00) notificationworker stdout | 2023-08-08 05:53:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:53:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 25, 372730), True, datetime.datetime(2023, 8, 8, 5, 53, 25, 372730), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:53:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:53:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:53:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:53:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:53:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:33.514094+00:00 (in 7.999188 seconds) exportactionlogsworker stdout | 2023-08-08 05:53:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:25 UTC)" (scheduled at 2023-08-08 05:53:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:53:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:53:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:53:25,958 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:53:26,487 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:53:26,689 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:53:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:53:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:49.687363+00:00 (in 22.796068 seconds) securityworker stdout | 2023-08-08 05:53:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:56 UTC)" (scheduled at 2023-08-08 05:53:26.890741+00:00) securityworker stdout | 2023-08-08 05:53:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:53:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:53:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:53:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:53:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:53:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:53:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:53:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:53:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:53:26,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:53:26,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:53:26,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:53:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:53:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:53:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:53:26,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:53:26,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:53:26,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:53:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:53:27,130 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:53:27,138 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:53:27,142 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:53:27,151 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:53:27,154 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:53:27,905 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:53:27,994 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:53:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:53:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:32.293757+00:00 (in 3.418327 seconds) chunkcleanupworker stdout | 2023-08-08 05:53:28,876 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:28 UTC)" (scheduled at 2023-08-08 05:53:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:53:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:53:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:53:29,097 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:53:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:53:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:39.417660+00:00 (in 9.999502 seconds) gcworker stdout | 2023-08-08 05:53:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:39 UTC)" (scheduled at 2023-08-08 05:53:29.417660+00:00) gcworker stdout | 2023-08-08 05:53:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264409418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:53:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:53:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:53:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:53:29,905 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:53:30,009 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:53:30,278 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:53:30,288 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:53:30,288 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:53:30,292 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:53:30,522 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:53:31,080 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:53:31,357 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:53:31,644 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:53:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:53:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:42.293757+00:00 (in 9.999436 seconds) chunkcleanupworker stdout | 2023-08-08 05:53:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:42 UTC)" (scheduled at 2023-08-08 05:53:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:53:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:53:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 32, 294818), True, datetime.datetime(2023, 8, 8, 5, 53, 32, 294818), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:53:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:53:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:53:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:53:32,932 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:53:32,937 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:53:32,943 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:53:32,946 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:53:32,949 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:53:32,953 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:53:32,956 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:53:32,959 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:53:32,962 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} exportactionlogsworker stdout | 2023-08-08 05:53:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:53:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:47.398551+00:00 (in 13.883872 seconds) exportactionlogsworker stdout | 2023-08-08 05:53:33,514 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:58:33 UTC)" (scheduled at 2023-08-08 05:53:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 05:53:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 53, 33, 515072), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:53:33,523 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 53, 33, 515072), True, datetime.datetime(2023, 8, 8, 5, 53, 33, 515072), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:53:33,524 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 53, 33, 515072), True, datetime.datetime(2023, 8, 8, 5, 53, 33, 515072), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 5, 53, 33, 515072), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:53:33,526 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:53:33,526 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:58:33 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:53:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:53:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:45.371682+00:00 (in 9.999465 seconds) notificationworker stdout | 2023-08-08 05:53:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:45 UTC)" (scheduled at 2023-08-08 05:53:35.371682+00:00) notificationworker stdout | 2023-08-08 05:53:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:53:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 35, 372640), True, datetime.datetime(2023, 8, 8, 5, 53, 35, 372640), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:53:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:53:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:53:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:53:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:53:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:54.393627+00:00 (in 17.813185 seconds) namespacegcworker stdout | 2023-08-08 05:53:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:36 UTC)" (scheduled at 2023-08-08 05:53:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:53:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:53:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:53:37,181 [197] [DEBUG] [app] Starting request: urn:request:31f42b10-1f41-4f0a-aef2-7cbd78a066b7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:53:37,181 [198] [DEBUG] [app] Starting request: urn:request:afc51bad-944b-4b5b-8bb3-355566330c53 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:53:37,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:37,183 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:37,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:37,186 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:53:37,187 [209] [DEBUG] [app] Starting request: urn:request:84697d0d-ada5-46ea-85c4-6d7c15a77c7a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:53:37,187 [209] [DEBUG] [app] Ending request: urn:request:84697d0d-ada5-46ea-85c4-6d7c15a77c7a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:84697d0d-ada5-46ea-85c4-6d7c15a77c7a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:53:37,187 [206] [DEBUG] [app] Starting request: urn:request:406e59b9-370a-466b-bf1d-e487d1bfac10 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:53:37,188 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:53:37,188 [206] [DEBUG] [app] Ending request: urn:request:406e59b9-370a-466b-bf1d-e487d1bfac10 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:406e59b9-370a-466b-bf1d-e487d1bfac10', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 05:53:37,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:53:37,188 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:53:37,188 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:37,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:37,190 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:37,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:37,193 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:37,193 [200] [DEBUG] [app] Starting request: urn:request:f11b2cf2-3ada-4bdd-9c1b-df6daaceb780 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:53:37,194 [198] [DEBUG] [app] Starting request: urn:request:0aca9fdb-c9a2-4f74-8075-c10a45bccd5a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:53:37,194 [198] [DEBUG] [app] Ending request: urn:request:0aca9fdb-c9a2-4f74-8075-c10a45bccd5a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0aca9fdb-c9a2-4f74-8075-c10a45bccd5a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:53:37,194 [200] [DEBUG] [app] Ending request: urn:request:f11b2cf2-3ada-4bdd-9c1b-df6daaceb780 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f11b2cf2-3ada-4bdd-9c1b-df6daaceb780', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:53:37,195 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2023-08-08 05:53:37,195 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:37,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:53:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:53:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:07.197412+00:00 (in 29.999487 seconds) buildlogsarchiver stdout | 2023-08-08 05:53:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:07 UTC)" (scheduled at 2023-08-08 05:53:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:53:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 53, 37, 198387), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:53:37,201 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:53:37,208 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:53:37,208 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:37,208 [197] [DEBUG] [app] Ending request: urn:request:31f42b10-1f41-4f0a-aef2-7cbd78a066b7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:31f42b10-1f41-4f0a-aef2-7cbd78a066b7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 05:53:37,208 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:53:37,208 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:37,209 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:53:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:53:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.029) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.115 159 0.115) gunicorn-web stdout | 2023-08-08 05:53:37,308 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:53:37,309 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:37,309 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:53:37,314 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:53:37,320 [198] [DEBUG] [app] Ending request: urn:request:afc51bad-944b-4b5b-8bb3-355566330c53 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:afc51bad-944b-4b5b-8bb3-355566330c53', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:53:37,320 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:37,321 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:53:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:53:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.141 47 0.140) gcworker stdout | 2023-08-08 05:53:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:53:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:49.417660+00:00 (in 9.999424 seconds) gcworker stdout | 2023-08-08 05:53:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:49 UTC)" (scheduled at 2023-08-08 05:53:39.417660+00:00) gcworker stdout | 2023-08-08 05:53:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264419418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:53:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:53:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:53:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:49 UTC)" executed successfully builder stdout | 2023-08-08 05:53:40,301 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:53:40,311 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:53:40,311 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:53:40,316 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:53:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:53:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:52.293757+00:00 (in 9.999505 seconds) chunkcleanupworker stdout | 2023-08-08 05:53:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:52 UTC)" (scheduled at 2023-08-08 05:53:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:53:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:53:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 42, 294752), True, datetime.datetime(2023, 8, 8, 5, 53, 42, 294752), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:53:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:53:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:53:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:53:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:53:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:49.178248+00:00 (in 3.806037 seconds) notificationworker stdout | 2023-08-08 05:53:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:55 UTC)" (scheduled at 2023-08-08 05:53:45.371682+00:00) notificationworker stdout | 2023-08-08 05:53:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:53:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 45, 372601), True, datetime.datetime(2023, 8, 8, 5, 53, 45, 372601), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:53:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:53:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:53:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:53:47,144 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:53:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:53:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:25.514341+00:00 (in 38.115247 seconds) exportactionlogsworker stdout | 2023-08-08 05:53:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:47 UTC)" (scheduled at 2023-08-08 05:53:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:53:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:53:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 47, 399452), True, datetime.datetime(2023, 8, 8, 5, 53, 47, 399452), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:53:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:53:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:53:47,409 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:53:48,329 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} notificationworker stdout | 2023-08-08 05:53:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:53:49,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:55.371682+00:00 (in 6.192864 seconds) notificationworker stdout | 2023-08-08 05:53:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:58:49 UTC)" (scheduled at 2023-08-08 05:53:49.178248+00:00) notificationworker stdout | 2023-08-08 05:53:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 53, 49, 179187), 'notification/%']) notificationworker stdout | 2023-08-08 05:53:49,187 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 53, 49, 179187), True, datetime.datetime(2023, 8, 8, 5, 53, 49, 179187), 0, 'notification/%']) notificationworker stdout | 2023-08-08 05:53:49,189 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 53, 49, 179187), True, datetime.datetime(2023, 8, 8, 5, 53, 49, 179187), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 5, 53, 49, 179187), 'notification/%']) notificationworker stdout | 2023-08-08 05:53:49,190 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:53:49,190 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:58:49 UTC)" executed successfully gcworker stdout | 2023-08-08 05:53:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:53:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:59.417660+00:00 (in 9.999368 seconds) gcworker stdout | 2023-08-08 05:53:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:59 UTC)" (scheduled at 2023-08-08 05:53:49.417660+00:00) gcworker stdout | 2023-08-08 05:53:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264429418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:53:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:53:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:53:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:53:59 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:53:49,493 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} securityworker stdout | 2023-08-08 05:53:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:53:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:53:56.890741+00:00 (in 7.202809 seconds) securityworker stdout | 2023-08-08 05:53:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:19 UTC)" (scheduled at 2023-08-08 05:53:49.687363+00:00) securityworker stdout | 2023-08-08 05:53:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:53:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:53:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:53:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:53:49,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:53:49,700 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:53:49,700 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:19 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:53:49,916 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 05:53:50,047 [199] [DEBUG] [app] Starting request: urn:request:7b5ef591-6539-4315-ad2e-19cf8e12dd10 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:53:50,047 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:53:50,047 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:53:50,048 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:53:50,055 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:53:50,055 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:53:50,055 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:53:50,055 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:53:50,055 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:53:50,055 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:53:50,055 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:53:50,055 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:53:50,058 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:53:50,064 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:53:50,065 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:53:50,069 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:53:50,072 [199] [DEBUG] [app] Ending request: urn:request:7b5ef591-6539-4315-ad2e-19cf8e12dd10 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:7b5ef591-6539-4315-ad2e-19cf8e12dd10', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:53:50,073 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:50,073 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:53:50 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:53:50 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 866 0.028) builder stdout | 2023-08-08 05:53:50,322 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:53:50,331 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:53:50,331 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:53:50,335 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:52,179 [198] [DEBUG] [app] Starting request: urn:request:bfc1a927-2722-4e07-b634-c591b69465c3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:53:52,180 [199] [DEBUG] [app] Starting request: urn:request:b3a95070-a9fd-4ed6-b414-768281130a07 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:53:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:52,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:53:52,185 [209] [DEBUG] [app] Starting request: urn:request:6b59106b-15db-46ef-a7d1-09872cd11b95 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:53:52,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:53:52,185 [209] [DEBUG] [app] Ending request: urn:request:6b59106b-15db-46ef-a7d1-09872cd11b95 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6b59106b-15db-46ef-a7d1-09872cd11b95', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:53:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:53:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:53:52,186 [203] [DEBUG] [app] Starting request: urn:request:fbd7d69b-a583-44c6-8eda-50deacf86c1a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:53:52,187 [203] [DEBUG] [app] Ending request: urn:request:fbd7d69b-a583-44c6-8eda-50deacf86c1a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fbd7d69b-a583-44c6-8eda-50deacf86c1a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:53:52,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:53:52,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:52,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:53:52,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:52,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:53:52,191 [200] [DEBUG] [app] Starting request: urn:request:2e5ab72d-a87e-4d9d-b175-7cb8fbb3a199 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:53:52,192 [198] [DEBUG] [app] Starting request: urn:request:54feb726-fabc-4a16-850b-41551380be1d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:53:52,192 [200] [DEBUG] [app] Ending request: urn:request:2e5ab72d-a87e-4d9d-b175-7cb8fbb3a199 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2e5ab72d-a87e-4d9d-b175-7cb8fbb3a199', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:53:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:53:52,192 [198] [DEBUG] [app] Ending request: urn:request:54feb726-fabc-4a16-850b-41551380be1d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:54feb726-fabc-4a16-850b-41551380be1d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:53:52,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:53:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:53:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:53:52,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:52,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:53:52,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:53:52,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:53:52,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:53:52,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:53:52,204 [199] [DEBUG] [app] Ending request: urn:request:b3a95070-a9fd-4ed6-b414-768281130a07 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b3a95070-a9fd-4ed6-b414-768281130a07', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:53:52,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:52,204 [198] [DEBUG] [app] Ending request: urn:request:bfc1a927-2722-4e07-b634-c591b69465c3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bfc1a927-2722-4e07-b634-c591b69465c3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:53:52,205 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:53:52,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:53:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:53:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 05:53:52,205 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:53:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:53:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) chunkcleanupworker stdout | 2023-08-08 05:53:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:53:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:02.293757+00:00 (in 9.999458 seconds) chunkcleanupworker stdout | 2023-08-08 05:53:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:02 UTC)" (scheduled at 2023-08-08 05:53:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:53:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:53:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 52, 294871), True, datetime.datetime(2023, 8, 8, 5, 53, 52, 294871), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:53:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:53:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:53:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:53:52,739 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:53:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:53:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:13.097375+00:00 (in 18.812723 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:53:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:54 UTC)" (scheduled at 2023-08-08 05:53:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:53:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:53:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:53:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:53:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:36.579841+00:00 (in 42.185686 seconds) namespacegcworker stdout | 2023-08-08 05:53:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:54 UTC)" (scheduled at 2023-08-08 05:53:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:53:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:53:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 54, 394724), True, datetime.datetime(2023, 8, 8, 5, 53, 54, 394724), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:53:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:53:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:53:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:54:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:53:54,507 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:53:54,582 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:53:54,734 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:53:54,737 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:53:54,740 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:53:54,853 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:53:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:53:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:02.178530+00:00 (in 6.806296 seconds) notificationworker stdout | 2023-08-08 05:53:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:05 UTC)" (scheduled at 2023-08-08 05:53:55.371682+00:00) notificationworker stdout | 2023-08-08 05:53:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:53:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 53, 55, 372670), True, datetime.datetime(2023, 8, 8, 5, 53, 55, 372670), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:53:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:53:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:53:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:53:55,996 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:53:56,521 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:53:56,713 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:53:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:53:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:19.687363+00:00 (in 22.796079 seconds) securityworker stdout | 2023-08-08 05:53:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:26 UTC)" (scheduled at 2023-08-08 05:53:56.890741+00:00) securityworker stdout | 2023-08-08 05:53:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:53:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:53:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:53:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:53:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:53:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:53:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:53:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:53:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:53:56,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:53:56,904 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:53:56,904 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:53:56,904 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:53:56,904 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:53:56,904 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:53:56,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:53:56,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:53:56,905 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:53:57,140 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:53:57,149 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:53:57,154 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:53:57,163 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:53:57,166 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:53:57,932 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:53:58,031 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:53:59,135 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:53:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:53:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:09.417660+00:00 (in 9.999477 seconds) gcworker stdout | 2023-08-08 05:53:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:09 UTC)" (scheduled at 2023-08-08 05:53:59.417660+00:00) gcworker stdout | 2023-08-08 05:53:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264439418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:53:59,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:53:59,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:53:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:53:59,942 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:54:00,046 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:54:00,343 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:54:00,353 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:54:00,353 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:54:00,357 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:54:00,558 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:54:01,102 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:54:01,384 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:54:01,679 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:54:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:54:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:05.371682+00:00 (in 3.192616 seconds) notificationworker stdout | 2023-08-08 05:54:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:02 UTC)" (scheduled at 2023-08-08 05:54:02.178530+00:00) notificationworker stdout | 2023-08-08 05:54:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:54:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:54:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:54:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:12.293757+00:00 (in 9.999492 seconds) chunkcleanupworker stdout | 2023-08-08 05:54:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:12 UTC)" (scheduled at 2023-08-08 05:54:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:54:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:54:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 2, 294801), True, datetime.datetime(2023, 8, 8, 5, 54, 2, 294801), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:54:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:54:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:54:02,942 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:54:02,947 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:54:02,954 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:54:02,957 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:54:02,961 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:54:02,964 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:54:02,967 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:54:02,970 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:54:02,973 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:54:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:54:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:09.793658+00:00 (in 5.605609 seconds) repositorygcworker stdout | 2023-08-08 05:54:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:04 UTC)" (scheduled at 2023-08-08 05:54:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:54:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:54:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:54:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:54:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:15.371682+00:00 (in 9.999488 seconds) notificationworker stdout | 2023-08-08 05:54:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:15 UTC)" (scheduled at 2023-08-08 05:54:05.371682+00:00) notificationworker stdout | 2023-08-08 05:54:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:54:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 5, 372547), True, datetime.datetime(2023, 8, 8, 5, 54, 5, 372547), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:54:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:54:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:54:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:54:07,178 [197] [DEBUG] [app] Starting request: urn:request:0f487cc3-eb0a-442a-a4ff-b30587cd9dc3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:54:07,180 [200] [DEBUG] [app] Starting request: urn:request:4b5d838b-186c-4787-b857-42ebbab20dfc (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:54:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:54:07,184 [203] [DEBUG] [app] Starting request: urn:request:0b09653c-4b5c-43d9-acc6-8034d957d24d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:54:07,185 [203] [DEBUG] [app] Ending request: urn:request:0b09653c-4b5c-43d9-acc6-8034d957d24d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0b09653c-4b5c-43d9-acc6-8034d957d24d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:54:07,185 [209] [DEBUG] [app] Starting request: urn:request:b1cc7f77-d5b2-45b4-ba43-3d827bfee78e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:54:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:54:07,186 [209] [DEBUG] [app] Ending request: urn:request:b1cc7f77-d5b2-45b4-ba43-3d827bfee78e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b1cc7f77-d5b2-45b4-ba43-3d827bfee78e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:54:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:07,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:07,191 [200] [DEBUG] [app] Starting request: urn:request:ffc2abd1-ca65-4d65-a7b8-81dda9df7dda (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:54:07,192 [200] [DEBUG] [app] Ending request: urn:request:ffc2abd1-ca65-4d65-a7b8-81dda9df7dda (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ffc2abd1-ca65-4d65-a7b8-81dda9df7dda', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:54:07,192 [199] [DEBUG] [app] Starting request: urn:request:73f230ab-823e-48d0-af50-e2c19c0e97b5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:54:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:07,193 [199] [DEBUG] [app] Ending request: urn:request:73f230ab-823e-48d0-af50-e2c19c0e97b5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:73f230ab-823e-48d0-af50-e2c19c0e97b5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:54:07,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:54:07,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:54:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:54:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:37.197412+00:00 (in 29.999476 seconds) buildlogsarchiver stdout | 2023-08-08 05:54:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:37 UTC)" (scheduled at 2023-08-08 05:54:07.197412+00:00) gunicorn-web stdout | 2023-08-08 05:54:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:54:07,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:54:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 54, 7, 198450), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:54:07,203 [200] [DEBUG] [app] Ending request: urn:request:4b5d838b-186c-4787-b857-42ebbab20dfc (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4b5d838b-186c-4787-b857-42ebbab20dfc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:54:07,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:54:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:54:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 05:54:07,205 [197] [DEBUG] [app] Ending request: urn:request:0f487cc3-eb0a-442a-a4ff-b30587cd9dc3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0f487cc3-eb0a-442a-a4ff-b30587cd9dc3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:54:07,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:07,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:54:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:54:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.029) buildlogsarchiver stdout | 2023-08-08 05:54:07,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:54:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:54:07,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:37 UTC)" executed successfully gcworker stdout | 2023-08-08 05:54:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:54:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:19.417660+00:00 (in 9.999478 seconds) gcworker stdout | 2023-08-08 05:54:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:19 UTC)" (scheduled at 2023-08-08 05:54:09.417660+00:00) gcworker stdout | 2023-08-08 05:54:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264449418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:54:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:54:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:54:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:54:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:54:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:19.187230+00:00 (in 9.392957 seconds) repositorygcworker stdout | 2023-08-08 05:54:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:09 UTC)" (scheduled at 2023-08-08 05:54:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:54:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:54:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 9, 794951), True, datetime.datetime(2023, 8, 8, 5, 54, 9, 794951), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:54:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:54:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:54:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:09 UTC)" executed successfully builder stdout | 2023-08-08 05:54:10,359 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:54:10,368 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:54:10,368 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:54:10,372 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:54:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:22.293757+00:00 (in 9.999483 seconds) chunkcleanupworker stdout | 2023-08-08 05:54:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:22 UTC)" (scheduled at 2023-08-08 05:54:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:54:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:54:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 12, 294768), True, datetime.datetime(2023, 8, 8, 5, 54, 12, 294768), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:54:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:54:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:12,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:54:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:54:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:54.284093+00:00 (in 41.186176 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:54:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:13 UTC)" (scheduled at 2023-08-08 05:54:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:54:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:54:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 13, 98419), True, datetime.datetime(2023, 8, 8, 5, 54, 13, 98419), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:54:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:54:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:54:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:54:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:54:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:25.371682+00:00 (in 9.999480 seconds) notificationworker stdout | 2023-08-08 05:54:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:25 UTC)" (scheduled at 2023-08-08 05:54:15.371682+00:00) notificationworker stdout | 2023-08-08 05:54:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:54:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 15, 372554), True, datetime.datetime(2023, 8, 8, 5, 54, 15, 372554), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:54:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:54:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:54:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:54:17,154 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:54:18,364 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} repositorygcworker stdout | 2023-08-08 05:54:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:54:19,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:04.187429+00:00 (in 44.999563 seconds) repositorygcworker stdout | 2023-08-08 05:54:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:59:19 UTC)" (scheduled at 2023-08-08 05:54:19.187230+00:00) repositorygcworker stdout | 2023-08-08 05:54:19,189 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 54, 19, 188461), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:54:19,196 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 54, 19, 188461), True, datetime.datetime(2023, 8, 8, 5, 54, 19, 188461), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:54:19,198 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 54, 19, 188461), True, datetime.datetime(2023, 8, 8, 5, 54, 19, 188461), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 5, 54, 19, 188461), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:54:19,199 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:54:19,199 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:59:19 UTC)" executed successfully gcworker stdout | 2023-08-08 05:54:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:54:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:29.417660+00:00 (in 9.999470 seconds) gcworker stdout | 2023-08-08 05:54:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:29 UTC)" (scheduled at 2023-08-08 05:54:19.417660+00:00) gcworker stdout | 2023-08-08 05:54:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264459418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:54:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:54:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:54:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:29 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:54:19,524 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} securityworker stdout | 2023-08-08 05:54:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:54:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:26.890741+00:00 (in 7.202900 seconds) securityworker stdout | 2023-08-08 05:54:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:49 UTC)" (scheduled at 2023-08-08 05:54:19.687363+00:00) securityworker stdout | 2023-08-08 05:54:19,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:54:19,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:54:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:54:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:54:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:54:19,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:54:19,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:49 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:54:19,929 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:54:20,382 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:54:20,392 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:54:20,392 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:54:20,396 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:21,168 [197] [DEBUG] [app] Starting request: urn:request:7aae70cf-d004-4ce6-8de4-50cb0671d320 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:54:21,169 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:54:21,169 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:54:21,169 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:54:21,176 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:54:21,176 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:54:21,176 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:54:21,176 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:54:21,177 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:54:21,177 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:54:21,177 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:54:21,177 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:54:21,179 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:54:21,185 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:54:21,186 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:54:21,190 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:54:21,193 [197] [DEBUG] [app] Ending request: urn:request:7aae70cf-d004-4ce6-8de4-50cb0671d320 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:7aae70cf-d004-4ce6-8de4-50cb0671d320', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:54:21,194 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:21,195 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:54:21 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:54:21 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 867 0.028) gunicorn-web stdout | 2023-08-08 05:54:22,179 [199] [DEBUG] [app] Starting request: urn:request:1733f288-cc10-45bb-b3cb-1caf2a7b66d2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:54:22,180 [200] [DEBUG] [app] Starting request: urn:request:067938fe-8472-471d-ba8a-4e01eb950f8f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:54:22,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:22,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:54:22,185 [208] [DEBUG] [app] Starting request: urn:request:1f3c957d-6c71-4a1d-bac9-394049f9dbc6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:54:22,185 [207] [DEBUG] [app] Starting request: urn:request:e69e81ac-4feb-46de-9294-45fc6e2183db (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:54:22,185 [208] [DEBUG] [app] Ending request: urn:request:1f3c957d-6c71-4a1d-bac9-394049f9dbc6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1f3c957d-6c71-4a1d-bac9-394049f9dbc6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:54:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:54:22,186 [207] [DEBUG] [app] Ending request: urn:request:e69e81ac-4feb-46de-9294-45fc6e2183db (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e69e81ac-4feb-46de-9294-45fc6e2183db', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:54:22,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:22,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:22,191 [200] [DEBUG] [app] Starting request: urn:request:3dd7337e-0b70-4b72-9b8a-22dfeb4f0180 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:54:22,191 [200] [DEBUG] [app] Ending request: urn:request:3dd7337e-0b70-4b72-9b8a-22dfeb4f0180 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3dd7337e-0b70-4b72-9b8a-22dfeb4f0180', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:54:22,192 [197] [DEBUG] [app] Starting request: urn:request:8e96ba32-a302-4f64-a51d-1a4a4f9440e1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:54:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:22,192 [197] [DEBUG] [app] Ending request: urn:request:8e96ba32-a302-4f64-a51d-1a4a4f9440e1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8e96ba32-a302-4f64-a51d-1a4a4f9440e1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:22,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:54:22,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:22,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:54:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:54:22,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:54:22,203 [200] [DEBUG] [app] Ending request: urn:request:067938fe-8472-471d-ba8a-4e01eb950f8f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:067938fe-8472-471d-ba8a-4e01eb950f8f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:54:22,203 [199] [DEBUG] [app] Ending request: urn:request:1733f288-cc10-45bb-b3cb-1caf2a7b66d2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1733f288-cc10-45bb-b3cb-1caf2a7b66d2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:54:22,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:22,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:54:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:54:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:54:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 05:54:22,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:54:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:54:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:54:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:28.874906+00:00 (in 6.580594 seconds) chunkcleanupworker stdout | 2023-08-08 05:54:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:32 UTC)" (scheduled at 2023-08-08 05:54:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:54:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:54:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 22, 294901), True, datetime.datetime(2023, 8, 8, 5, 54, 22, 294901), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:54:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:54:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:54:22,757 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:54:24,533 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:54:24,592 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:54:24,742 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:54:24,746 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:54:24,749 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:54:24,885 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:54:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:54:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:35.371682+00:00 (in 9.999493 seconds) notificationworker stdout | 2023-08-08 05:54:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:35 UTC)" (scheduled at 2023-08-08 05:54:25.371682+00:00) notificationworker stdout | 2023-08-08 05:54:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:54:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 25, 372571), True, datetime.datetime(2023, 8, 8, 5, 54, 25, 372571), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:54:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:54:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:54:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:54:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:54:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:47.398551+00:00 (in 21.883639 seconds) exportactionlogsworker stdout | 2023-08-08 05:54:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:25 UTC)" (scheduled at 2023-08-08 05:54:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:54:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:54:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:54:26,033 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:54:26,540 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:54:26,729 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:54:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:54:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:49.687363+00:00 (in 22.796074 seconds) securityworker stdout | 2023-08-08 05:54:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:56 UTC)" (scheduled at 2023-08-08 05:54:26.890741+00:00) securityworker stdout | 2023-08-08 05:54:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:54:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:54:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:54:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:54:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:54:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:54:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:54:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:54:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:54:26,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:54:26,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:54:26,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:54:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:54:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:54:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:54:26,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:54:26,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:54:26,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:54:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:54:27,148 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:54:27,160 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:54:27,165 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:54:27,175 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:54:27,178 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:54:27,939 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:54:28,068 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:54:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:54:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:32.293757+00:00 (in 3.418301 seconds) chunkcleanupworker stdout | 2023-08-08 05:54:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:28 UTC)" (scheduled at 2023-08-08 05:54:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:54:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:54:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:54:29,171 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:54:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:54:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:39.417660+00:00 (in 9.999516 seconds) gcworker stdout | 2023-08-08 05:54:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:39 UTC)" (scheduled at 2023-08-08 05:54:29.417660+00:00) gcworker stdout | 2023-08-08 05:54:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264469418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:54:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:54:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:54:29,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:54:29,980 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:54:30,084 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:54:30,400 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:54:30,409 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:54:30,409 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:54:30,413 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:54:30,578 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:54:31,140 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:54:31,406 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:54:31,716 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:54:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:54:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:42.293757+00:00 (in 9.999371 seconds) chunkcleanupworker stdout | 2023-08-08 05:54:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:42 UTC)" (scheduled at 2023-08-08 05:54:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:54:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:54:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 32, 294976), True, datetime.datetime(2023, 8, 8, 5, 54, 32, 294976), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:54:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:54:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:54:32,955 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:54:32,958 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:54:32,963 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:54:32,966 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:54:32,972 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:54:32,975 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:54:32,978 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:54:32,981 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:54:32,984 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 05:54:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:54:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:45.371682+00:00 (in 9.999462 seconds) notificationworker stdout | 2023-08-08 05:54:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:45 UTC)" (scheduled at 2023-08-08 05:54:35.371682+00:00) notificationworker stdout | 2023-08-08 05:54:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:54:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 35, 372638), True, datetime.datetime(2023, 8, 8, 5, 54, 35, 372638), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:54:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:54:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:54:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:54:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:54:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:39.579576+00:00 (in 2.999220 seconds) namespacegcworker stdout | 2023-08-08 05:54:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:36 UTC)" (scheduled at 2023-08-08 05:54:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:54:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:54:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:54:37,179 [197] [DEBUG] [app] Starting request: urn:request:1e946041-8e1e-4c27-8d84-64ff2e669657 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:54:37,181 [200] [DEBUG] [app] Starting request: urn:request:e89b1aed-9250-4f3c-b03d-b4b651d70acd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:54:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:54:37,185 [210] [DEBUG] [app] Starting request: urn:request:860ebf64-ac9a-402f-9fd9-71782d34a8a8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:54:37,186 [210] [DEBUG] [app] Ending request: urn:request:860ebf64-ac9a-402f-9fd9-71782d34a8a8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:860ebf64-ac9a-402f-9fd9-71782d34a8a8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:54:37,186 [209] [DEBUG] [app] Starting request: urn:request:d7264544-9354-481a-a345-7513ef360910 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:54:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:54:37,187 [209] [DEBUG] [app] Ending request: urn:request:d7264544-9354-481a-a345-7513ef360910 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d7264544-9354-481a-a345-7513ef360910', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:54:37,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:37,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:37,191 [197] [DEBUG] [app] Starting request: urn:request:103346a2-d2db-47af-9c27-1251dc0e4f93 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:54:37,192 [197] [DEBUG] [app] Ending request: urn:request:103346a2-d2db-47af-9c27-1251dc0e4f93 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:103346a2-d2db-47af-9c27-1251dc0e4f93', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:37,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:54:37,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:54:37,193 [199] [DEBUG] [app] Starting request: urn:request:8a4fd1e6-ff86-45a0-8b92-71934cc164d4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:54:37,194 [199] [DEBUG] [app] Ending request: urn:request:8a4fd1e6-ff86-45a0-8b92-71934cc164d4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8a4fd1e6-ff86-45a0-8b92-71934cc164d4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:54:37,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:37,195 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:37,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:54:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:54:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:07.197412+00:00 (in 29.999517 seconds) buildlogsarchiver stdout | 2023-08-08 05:54:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:07 UTC)" (scheduled at 2023-08-08 05:54:37.197412+00:00) gunicorn-web stdout | 2023-08-08 05:54:37,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:54:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 54, 37, 198399), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:54:37,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:54:37,204 [197] [DEBUG] [app] Ending request: urn:request:1e946041-8e1e-4c27-8d84-64ff2e669657 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1e946041-8e1e-4c27-8d84-64ff2e669657', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:54:37,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:54:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:54:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 05:54:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:54:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:54:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:54:37,205 [200] [DEBUG] [app] Ending request: urn:request:e89b1aed-9250-4f3c-b03d-b4b651d70acd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e89b1aed-9250-4f3c-b03d-b4b651d70acd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:54:37,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:54:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:54:37,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:54:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gcworker stdout | 2023-08-08 05:54:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:54:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:49.417660+00:00 (in 9.999443 seconds) gcworker stdout | 2023-08-08 05:54:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:49 UTC)" (scheduled at 2023-08-08 05:54:39.417660+00:00) gcworker stdout | 2023-08-08 05:54:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264479418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:54:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:54:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:54:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:54:39,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:54:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:54.393627+00:00 (in 14.813479 seconds) namespacegcworker stdout | 2023-08-08 05:54:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:59:39 UTC)" (scheduled at 2023-08-08 05:54:39.579576+00:00) namespacegcworker stdout | 2023-08-08 05:54:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 54, 39, 580686), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:54:39,588 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 54, 39, 580686), True, datetime.datetime(2023, 8, 8, 5, 54, 39, 580686), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:54:39,590 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 54, 39, 580686), True, datetime.datetime(2023, 8, 8, 5, 54, 39, 580686), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 5, 54, 39, 580686), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:54:39,591 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:54:39,591 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:59:39 UTC)" executed successfully builder stdout | 2023-08-08 05:54:40,423 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:54:40,431 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:54:40,432 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:54:40,436 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:54:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:46.874674+00:00 (in 4.580358 seconds) chunkcleanupworker stdout | 2023-08-08 05:54:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:52 UTC)" (scheduled at 2023-08-08 05:54:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:54:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:54:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 42, 294804), True, datetime.datetime(2023, 8, 8, 5, 54, 42, 294804), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:54:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:54:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:54:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:54:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:55.371682+00:00 (in 9.999488 seconds) notificationworker stdout | 2023-08-08 05:54:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:55 UTC)" (scheduled at 2023-08-08 05:54:45.371682+00:00) notificationworker stdout | 2023-08-08 05:54:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:54:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 45, 372560), True, datetime.datetime(2023, 8, 8, 5, 54, 45, 372560), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:54:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:54:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:54:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:54:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:54:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:52.293757+00:00 (in 5.418566 seconds) chunkcleanupworker stdout | 2023-08-08 05:54:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:59:46 UTC)" (scheduled at 2023-08-08 05:54:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 05:54:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 54, 46, 875658), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:54:46,882 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 54, 46, 875658), True, datetime.datetime(2023, 8, 8, 5, 54, 46, 875658), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:54:46,884 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 54, 46, 875658), True, datetime.datetime(2023, 8, 8, 5, 54, 46, 875658), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 5, 54, 46, 875658), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:54:46,885 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:46,885 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 05:59:46 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:54:47,192 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:54:47,399 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:54:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:25.514341+00:00 (in 38.115217 seconds) exportactionlogsworker stdout | 2023-08-08 05:54:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:47 UTC)" (scheduled at 2023-08-08 05:54:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:54:47,400 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:54:47,401 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 47, 400135), True, datetime.datetime(2023, 8, 8, 5, 54, 47, 400135), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:54:47,409 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:54:47,409 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:54:47,409 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:54:48,401 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:54:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:54:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:59.417660+00:00 (in 9.999488 seconds) gcworker stdout | 2023-08-08 05:54:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:59 UTC)" (scheduled at 2023-08-08 05:54:49.417660+00:00) gcworker stdout | 2023-08-08 05:54:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264489418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:54:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:54:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:54:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:54:59 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:54:49,553 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} securityworker stdout | 2023-08-08 05:54:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:54:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:54:56.890741+00:00 (in 7.202881 seconds) securityworker stdout | 2023-08-08 05:54:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:19 UTC)" (scheduled at 2023-08-08 05:54:49.687363+00:00) securityworker stdout | 2023-08-08 05:54:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:54:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:54:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:54:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:54:49,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:54:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:54:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:19 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:54:49,957 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:54:50,446 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:54:50,456 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:54:50,456 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:54:50,460 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:52,179 [197] [DEBUG] [app] Starting request: urn:request:730996a2-51ad-40ac-b311-290c71f1c259 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:54:52,179 [200] [DEBUG] [app] Starting request: urn:request:56e1349b-85ed-4e65-9d57-ba699b272a70 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:54:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:54:52,185 [203] [DEBUG] [app] Starting request: urn:request:e5a725c3-0b8f-4c7c-a769-0799ce3ac638 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:54:52,185 [203] [DEBUG] [app] Ending request: urn:request:e5a725c3-0b8f-4c7c-a769-0799ce3ac638 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e5a725c3-0b8f-4c7c-a769-0799ce3ac638', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:54:52,185 [210] [DEBUG] [app] Starting request: urn:request:fee3de4e-94f5-4deb-93f5-23f01100d3e0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:54:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:54:52,186 [210] [DEBUG] [app] Ending request: urn:request:fee3de4e-94f5-4deb-93f5-23f01100d3e0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fee3de4e-94f5-4deb-93f5-23f01100d3e0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:54:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:54:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:54:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:52,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:54:52,192 [200] [DEBUG] [app] Starting request: urn:request:8aa5c78f-a97e-4069-bfe1-a19bb64f83c1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:54:52,192 [200] [DEBUG] [app] Ending request: urn:request:8aa5c78f-a97e-4069-bfe1-a19bb64f83c1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8aa5c78f-a97e-4069-bfe1-a19bb64f83c1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:52,192 [198] [DEBUG] [app] Starting request: urn:request:be6eb035-8a13-426d-890d-6375ea21ff6f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:54:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:54:52,193 [198] [DEBUG] [app] Ending request: urn:request:be6eb035-8a13-426d-890d-6375ea21ff6f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:be6eb035-8a13-426d-890d-6375ea21ff6f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:54:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:54:52,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:54:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:54:52,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:54:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:54:52,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:54:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:54:52,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:54:52,204 [200] [DEBUG] [app] Ending request: urn:request:56e1349b-85ed-4e65-9d57-ba699b272a70 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:56e1349b-85ed-4e65-9d57-ba699b272a70', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:54:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:52,204 [197] [DEBUG] [app] Ending request: urn:request:730996a2-51ad-40ac-b311-290c71f1c259 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:730996a2-51ad-40ac-b311-290c71f1c259', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:54:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:54:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:54:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:54:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:54:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:54:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:54:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) chunkcleanupworker stdout | 2023-08-08 05:54:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:54:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:02.293757+00:00 (in 9.999459 seconds) chunkcleanupworker stdout | 2023-08-08 05:54:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:02 UTC)" (scheduled at 2023-08-08 05:54:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:54:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:54:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 52, 294918), True, datetime.datetime(2023, 8, 8, 5, 54, 52, 294918), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:54:52,340 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:54:52,341 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:54:52,341 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:54:52,775 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:54:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:54:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:13.097375+00:00 (in 18.812706 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:54:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:54 UTC)" (scheduled at 2023-08-08 05:54:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:54:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:54:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:54:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:54:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:36.579841+00:00 (in 42.185618 seconds) namespacegcworker stdout | 2023-08-08 05:54:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:54 UTC)" (scheduled at 2023-08-08 05:54:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:54:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:54:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 54, 394784), True, datetime.datetime(2023, 8, 8, 5, 54, 54, 394784), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:54:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:54:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:54:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:55:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:54:54,544 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:54:54,629 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:54:54,750 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:54:54,753 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:54:54,756 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:54:54,915 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:54:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:54:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:02.178530+00:00 (in 6.806358 seconds) notificationworker stdout | 2023-08-08 05:54:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:05 UTC)" (scheduled at 2023-08-08 05:54:55.371682+00:00) notificationworker stdout | 2023-08-08 05:54:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:54:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 54, 55, 372511), True, datetime.datetime(2023, 8, 8, 5, 54, 55, 372511), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:54:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:54:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:54:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:54:56,040 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:54:56,555 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:54:56,745 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:54:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:54:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:19.687363+00:00 (in 22.796124 seconds) securityworker stdout | 2023-08-08 05:54:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:26 UTC)" (scheduled at 2023-08-08 05:54:56.890741+00:00) securityworker stdout | 2023-08-08 05:54:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:54:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:54:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:54:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:54:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:54:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:54:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:54:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:54:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:54:56,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:54:56,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:54:56,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:54:56,902 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:54:56,902 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:54:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:54:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:54:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:54:56,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:54:57,157 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:54:57,170 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:54:57,176 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:54:57,186 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:54:57,189 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:54:57,947 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:54:58,105 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:54:59,183 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:54:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:54:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:09.417660+00:00 (in 9.999501 seconds) gcworker stdout | 2023-08-08 05:54:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:09 UTC)" (scheduled at 2023-08-08 05:54:59.417660+00:00) gcworker stdout | 2023-08-08 05:54:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264499418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:54:59,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:54:59,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:54:59,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:55:00,016 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:55:00,120 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:55:00,470 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:55:00,480 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:55:00,480 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:55:00,484 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:55:00,610 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:55:01,165 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:55:01,416 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:55:01,746 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:55:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:55:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:05.371682+00:00 (in 3.192599 seconds) notificationworker stdout | 2023-08-08 05:55:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:02 UTC)" (scheduled at 2023-08-08 05:55:02.178530+00:00) notificationworker stdout | 2023-08-08 05:55:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:55:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:55:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:55:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:12.293757+00:00 (in 9.999478 seconds) chunkcleanupworker stdout | 2023-08-08 05:55:02,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:12 UTC)" (scheduled at 2023-08-08 05:55:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:55:02,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:55:02,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 2, 295185), True, datetime.datetime(2023, 8, 8, 5, 55, 2, 295185), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:55:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:55:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:55:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:55:02,967 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:55:02,969 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:55:02,972 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:55:02,976 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:55:02,981 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:55:02,983 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:55:02,988 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:55:02,991 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:55:02,995 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:55:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:55:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:09.793658+00:00 (in 5.605712 seconds) repositorygcworker stdout | 2023-08-08 05:55:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:04 UTC)" (scheduled at 2023-08-08 05:55:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:55:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:55:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:55:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:55:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:15.371682+00:00 (in 9.999379 seconds) notificationworker stdout | 2023-08-08 05:55:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:15 UTC)" (scheduled at 2023-08-08 05:55:05.371682+00:00) notificationworker stdout | 2023-08-08 05:55:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:55:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 5, 372800), True, datetime.datetime(2023, 8, 8, 5, 55, 5, 372800), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:55:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:55:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:55:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:55:07,179 [200] [DEBUG] [app] Starting request: urn:request:c713c7a2-508d-4ee6-82eb-93279e035f0c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:55:07,179 [199] [DEBUG] [app] Starting request: urn:request:523e974b-1a19-4c9c-837a-7d182b11a356 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:55:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:07,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:55:07,185 [203] [DEBUG] [app] Starting request: urn:request:bf7c8e49-b848-432f-b173-f67c0894fe21 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:55:07,185 [210] [DEBUG] [app] Starting request: urn:request:2910f3d1-f628-4237-bcbb-bac83f12e259 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:55:07,186 [210] [DEBUG] [app] Ending request: urn:request:2910f3d1-f628-4237-bcbb-bac83f12e259 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2910f3d1-f628-4237-bcbb-bac83f12e259', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:55:07,186 [203] [DEBUG] [app] Ending request: urn:request:bf7c8e49-b848-432f-b173-f67c0894fe21 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:bf7c8e49-b848-432f-b173-f67c0894fe21', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:55:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:55:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 05:55:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:07,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:07,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:07,192 [197] [DEBUG] [app] Starting request: urn:request:62a43205-372d-4f59-8333-d6f03c945955 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:55:07,193 [197] [DEBUG] [app] Ending request: urn:request:62a43205-372d-4f59-8333-d6f03c945955 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:62a43205-372d-4f59-8333-d6f03c945955', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:07,193 [200] [DEBUG] [app] Starting request: urn:request:0c40fbaf-0890-4298-b57d-67e47913724e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:55:07,194 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2023-08-08 05:55:07,194 [200] [DEBUG] [app] Ending request: urn:request:0c40fbaf-0890-4298-b57d-67e47913724e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0c40fbaf-0890-4298-b57d-67e47913724e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:07,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:55:07,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2023-08-08 05:55:07,195 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:07,195 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:55:07,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:55:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:55:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:37.197412+00:00 (in 29.999457 seconds) buildlogsarchiver stdout | 2023-08-08 05:55:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:37 UTC)" (scheduled at 2023-08-08 05:55:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:55:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 55, 7, 198566), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:55:07,200 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:55:07,200 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:55:07,205 [199] [DEBUG] [app] Ending request: urn:request:523e974b-1a19-4c9c-837a-7d182b11a356 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:523e974b-1a19-4c9c-837a-7d182b11a356', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:55:07,205 [200] [DEBUG] [app] Ending request: urn:request:c713c7a2-508d-4ee6-82eb-93279e035f0c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c713c7a2-508d-4ee6-82eb-93279e035f0c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:55:07,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:07,206 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:07,206 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:55:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:55:07,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:55:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:55:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:55:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) buildlogsarchiver stdout | 2023-08-08 05:55:07,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:55:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:55:07,207 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:37 UTC)" executed successfully gcworker stdout | 2023-08-08 05:55:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:55:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:19.417660+00:00 (in 9.999443 seconds) gcworker stdout | 2023-08-08 05:55:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:19 UTC)" (scheduled at 2023-08-08 05:55:09.417660+00:00) gcworker stdout | 2023-08-08 05:55:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264509418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:55:09,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:55:09,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:55:09,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:55:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:55:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:04.187429+00:00 (in 54.393227 seconds) repositorygcworker stdout | 2023-08-08 05:55:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:09 UTC)" (scheduled at 2023-08-08 05:55:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:55:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:55:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 9, 794806), True, datetime.datetime(2023, 8, 8, 5, 55, 9, 794806), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:55:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:55:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:55:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:09 UTC)" executed successfully builder stdout | 2023-08-08 05:55:10,486 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:55:10,494 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:55:10,494 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:55:10,499 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:55:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:55:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:22.293757+00:00 (in 9.999491 seconds) chunkcleanupworker stdout | 2023-08-08 05:55:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:22 UTC)" (scheduled at 2023-08-08 05:55:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:55:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:55:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 12, 294753), True, datetime.datetime(2023, 8, 8, 5, 55, 12, 294753), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:55:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:55:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:55:12,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:55:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:55:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:54.284093+00:00 (in 41.186198 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:55:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:13 UTC)" (scheduled at 2023-08-08 05:55:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:55:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:55:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 13, 98410), True, datetime.datetime(2023, 8, 8, 5, 55, 13, 98410), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:55:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:55:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:55:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:55:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:55:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:25.371682+00:00 (in 9.999488 seconds) notificationworker stdout | 2023-08-08 05:55:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:25 UTC)" (scheduled at 2023-08-08 05:55:15.371682+00:00) notificationworker stdout | 2023-08-08 05:55:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:55:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 15, 372688), True, datetime.datetime(2023, 8, 8, 5, 55, 15, 372688), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:55:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:55:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:55:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:55:17,229 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:55:18,438 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:55:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:55:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:29.417660+00:00 (in 9.999488 seconds) gcworker stdout | 2023-08-08 05:55:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:29 UTC)" (scheduled at 2023-08-08 05:55:19.417660+00:00) gcworker stdout | 2023-08-08 05:55:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264519418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:55:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:55:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:55:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:29 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:55:19,590 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} securityworker stdout | 2023-08-08 05:55:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:55:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:26.890741+00:00 (in 7.202895 seconds) securityworker stdout | 2023-08-08 05:55:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:49 UTC)" (scheduled at 2023-08-08 05:55:19.687363+00:00) securityworker stdout | 2023-08-08 05:55:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:55:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:55:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:55:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:55:19,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:55:19,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:55:19,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:49 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:55:19,994 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:55:20,509 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:55:20,518 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:55:20,518 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:55:20,523 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:22,179 [198] [DEBUG] [app] Starting request: urn:request:5adc7ef3-e56f-406a-a496-36e2c6e036ca (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:55:22,179 [199] [DEBUG] [app] Starting request: urn:request:009cee2b-0679-4b1b-9770-89b99d60a697 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:55:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:22,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:22,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:55:22,185 [203] [DEBUG] [app] Starting request: urn:request:95ef1848-9e7d-4ded-9104-2a7a58c93aad (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:55:22,185 [203] [DEBUG] [app] Ending request: urn:request:95ef1848-9e7d-4ded-9104-2a7a58c93aad (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:95ef1848-9e7d-4ded-9104-2a7a58c93aad', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:55:22,185 [206] [DEBUG] [app] Starting request: urn:request:61f4246d-6290-4576-8f80-dd6b3af22ad0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:55:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:55:22,186 [206] [DEBUG] [app] Ending request: urn:request:61f4246d-6290-4576-8f80-dd6b3af22ad0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:61f4246d-6290-4576-8f80-dd6b3af22ad0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:55:22,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:55:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:22,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:22,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:22,191 [197] [DEBUG] [app] Starting request: urn:request:5d436b8c-5926-4126-86da-8a0676af00ef (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:55:22,191 [199] [DEBUG] [app] Starting request: urn:request:b6e37c7a-72a2-4ee4-a7ee-54b40d844cea (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:55:22,191 [197] [DEBUG] [app] Ending request: urn:request:5d436b8c-5926-4126-86da-8a0676af00ef (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5d436b8c-5926-4126-86da-8a0676af00ef', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:22,191 [199] [DEBUG] [app] Ending request: urn:request:b6e37c7a-72a2-4ee4-a7ee-54b40d844cea (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b6e37c7a-72a2-4ee4-a7ee-54b40d844cea', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:55:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:55:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:55:22,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:22,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:55:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:55:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:55:22,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:55:22,202 [199] [DEBUG] [app] Ending request: urn:request:009cee2b-0679-4b1b-9770-89b99d60a697 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:009cee2b-0679-4b1b-9770-89b99d60a697', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:55:22,202 [198] [DEBUG] [app] Ending request: urn:request:5adc7ef3-e56f-406a-a496-36e2c6e036ca (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5adc7ef3-e56f-406a-a496-36e2c6e036ca', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:55:22,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:22,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:22,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:55:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:55:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:55:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 05:55:22,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:55:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:55:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:55:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:28.874906+00:00 (in 6.580575 seconds) chunkcleanupworker stdout | 2023-08-08 05:55:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:32 UTC)" (scheduled at 2023-08-08 05:55:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:55:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:55:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 22, 295015), True, datetime.datetime(2023, 8, 8, 5, 55, 22, 295015), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:55:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:55:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:55:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:55:22,789 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:55:24,577 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:55:24,645 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:55:24,759 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:55:24,761 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:55:24,764 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:55:24,934 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:55:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:55:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:35.371682+00:00 (in 9.999503 seconds) notificationworker stdout | 2023-08-08 05:55:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:35 UTC)" (scheduled at 2023-08-08 05:55:25.371682+00:00) notificationworker stdout | 2023-08-08 05:55:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:55:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 25, 372753), True, datetime.datetime(2023, 8, 8, 5, 55, 25, 372753), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:55:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:55:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:55:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:55:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:55:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:47.398551+00:00 (in 21.883690 seconds) exportactionlogsworker stdout | 2023-08-08 05:55:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:25 UTC)" (scheduled at 2023-08-08 05:55:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:55:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:55:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:55:26,077 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:55:26,591 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:55:26,781 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:55:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:55:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:49.687363+00:00 (in 22.796098 seconds) securityworker stdout | 2023-08-08 05:55:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:56 UTC)" (scheduled at 2023-08-08 05:55:26.890741+00:00) securityworker stdout | 2023-08-08 05:55:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:55:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:55:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:55:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:55:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:55:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:55:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:55:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:55:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:55:26,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:55:26,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:55:26,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:55:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:55:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:55:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:55:26,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:55:26,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:55:26,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:55:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:55:27,165 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:55:27,181 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:55:27,188 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:55:27,197 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:55:27,201 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:55:27,983 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:55:28,142 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:55:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:55:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:32.293757+00:00 (in 3.418199 seconds) chunkcleanupworker stdout | 2023-08-08 05:55:28,876 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:28 UTC)" (scheduled at 2023-08-08 05:55:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:55:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:55:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:55:29,219 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:55:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:55:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:39.417660+00:00 (in 9.999514 seconds) gcworker stdout | 2023-08-08 05:55:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:39 UTC)" (scheduled at 2023-08-08 05:55:29.417660+00:00) gcworker stdout | 2023-08-08 05:55:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264529418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:55:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:55:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:55:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:55:30,038 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:55:30,152 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:55:30,528 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:55:30,537 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:55:30,537 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:55:30,541 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:55:30,646 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:55:31,177 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:55:31,429 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:55:31,783 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:55:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:55:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:42.293757+00:00 (in 9.999361 seconds) chunkcleanupworker stdout | 2023-08-08 05:55:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:42 UTC)" (scheduled at 2023-08-08 05:55:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:55:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:55:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 32, 294905), True, datetime.datetime(2023, 8, 8, 5, 55, 32, 294905), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:55:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:55:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:55:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:55:32,977 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:55:32,979 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:55:32,983 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:55:32,985 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:55:32,989 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:55:32,992 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:55:32,999 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:55:33,001 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:55:33,004 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 05:55:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:55:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:45.371682+00:00 (in 9.999468 seconds) notificationworker stdout | 2023-08-08 05:55:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:45 UTC)" (scheduled at 2023-08-08 05:55:35.371682+00:00) notificationworker stdout | 2023-08-08 05:55:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:55:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 35, 372711), True, datetime.datetime(2023, 8, 8, 5, 55, 35, 372711), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:55:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:55:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:55:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:55:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:55:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:54.393627+00:00 (in 17.813213 seconds) namespacegcworker stdout | 2023-08-08 05:55:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:36 UTC)" (scheduled at 2023-08-08 05:55:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:55:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:55:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:55:37,179 [198] [DEBUG] [app] Starting request: urn:request:491ac7ac-5ac3-4205-b47f-766c9650ce76 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:55:37,179 [197] [DEBUG] [app] Starting request: urn:request:99e67412-7c1e-4e74-a049-14cc53118df3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:55:37,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:37,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:55:37,184 [208] [DEBUG] [app] Starting request: urn:request:3b489b10-ffc7-4ea2-bec1-c4797838d005 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:55:37,184 [209] [DEBUG] [app] Starting request: urn:request:c3e2cda4-b732-4752-b980-2fdbb50dafdc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:55:37,185 [208] [DEBUG] [app] Ending request: urn:request:3b489b10-ffc7-4ea2-bec1-c4797838d005 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3b489b10-ffc7-4ea2-bec1-c4797838d005', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:55:37,185 [209] [DEBUG] [app] Ending request: urn:request:c3e2cda4-b732-4752-b980-2fdbb50dafdc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c3e2cda4-b732-4752-b980-2fdbb50dafdc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:55:37,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:55:37,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:55:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:37,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:37,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:37,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:37,191 [198] [DEBUG] [app] Starting request: urn:request:350625bb-728c-4f2c-8d65-8768d886321d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:55:37,191 [198] [DEBUG] [app] Ending request: urn:request:350625bb-728c-4f2c-8d65-8768d886321d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:350625bb-728c-4f2c-8d65-8768d886321d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:37,191 [199] [DEBUG] [app] Starting request: urn:request:4934e65c-7a22-4a9d-a6d2-b8bbeacf3f72 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:55:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:55:37,192 [199] [DEBUG] [app] Ending request: urn:request:4934e65c-7a22-4a9d-a6d2-b8bbeacf3f72 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4934e65c-7a22-4a9d-a6d2-b8bbeacf3f72', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:37,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:55:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:55:37,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:55:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:55:37,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:55:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:55:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:55:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:07.197412+00:00 (in 29.999398 seconds) buildlogsarchiver stdout | 2023-08-08 05:55:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:07 UTC)" (scheduled at 2023-08-08 05:55:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:55:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 55, 37, 198691), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:55:37,203 [198] [DEBUG] [app] Ending request: urn:request:491ac7ac-5ac3-4205-b47f-766c9650ce76 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:491ac7ac-5ac3-4205-b47f-766c9650ce76', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:55:37,203 [197] [DEBUG] [app] Ending request: urn:request:99e67412-7c1e-4e74-a049-14cc53118df3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:99e67412-7c1e-4e74-a049-14cc53118df3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:55:37,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:37,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:55:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 05:55:37,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:55:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:55:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 05:55:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:55:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:55:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:55:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:55:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:07 UTC)" executed successfully gcworker stdout | 2023-08-08 05:55:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:55:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:49.417660+00:00 (in 9.999465 seconds) gcworker stdout | 2023-08-08 05:55:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:49 UTC)" (scheduled at 2023-08-08 05:55:39.417660+00:00) gcworker stdout | 2023-08-08 05:55:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264539418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:55:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:55:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:55:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:49 UTC)" executed successfully builder stdout | 2023-08-08 05:55:40,542 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:55:40,553 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:55:40,553 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:55:40,559 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:55:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:55:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:52.293757+00:00 (in 9.999474 seconds) chunkcleanupworker stdout | 2023-08-08 05:55:42,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:52 UTC)" (scheduled at 2023-08-08 05:55:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:55:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:55:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 42, 295262), True, datetime.datetime(2023, 8, 8, 5, 55, 42, 295262), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:55:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:55:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:55:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:55:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:55:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:55.371682+00:00 (in 9.999484 seconds) notificationworker stdout | 2023-08-08 05:55:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:55 UTC)" (scheduled at 2023-08-08 05:55:45.371682+00:00) notificationworker stdout | 2023-08-08 05:55:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:55:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 45, 372563), True, datetime.datetime(2023, 8, 8, 5, 55, 45, 372563), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:55:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:55:45,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:55:45,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:55:47,266 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:55:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:55:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:25.514341+00:00 (in 38.115212 seconds) exportactionlogsworker stdout | 2023-08-08 05:55:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:47 UTC)" (scheduled at 2023-08-08 05:55:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:55:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:55:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 47, 399714), True, datetime.datetime(2023, 8, 8, 5, 55, 47, 399714), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:55:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:55:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:55:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:55:48,474 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:55:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:55:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:59.417660+00:00 (in 9.999443 seconds) gcworker stdout | 2023-08-08 05:55:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:59 UTC)" (scheduled at 2023-08-08 05:55:49.417660+00:00) gcworker stdout | 2023-08-08 05:55:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264549418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:55:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:55:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:55:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:55:59 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:55:49,628 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} securityworker stdout | 2023-08-08 05:55:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:55:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:55:56.890741+00:00 (in 7.202868 seconds) securityworker stdout | 2023-08-08 05:55:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:19 UTC)" (scheduled at 2023-08-08 05:55:49.687363+00:00) securityworker stdout | 2023-08-08 05:55:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:55:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:55:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:55:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:55:49,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:55:49,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:55:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:19 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:55:50,002 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:55:50,568 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:55:50,577 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:55:50,577 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:55:50,582 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:52,178 [200] [DEBUG] [app] Starting request: urn:request:104379df-b9b1-4279-abfc-208763487840 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:55:52,179 [197] [DEBUG] [app] Starting request: urn:request:5389376c-0db0-42cf-b0c3-65f6f85da95d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:55:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:55:52,184 [203] [DEBUG] [app] Starting request: urn:request:98b1f11b-bf60-4758-b59d-18b2a0291dbf (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:55:52,185 [210] [DEBUG] [app] Starting request: urn:request:7478cafc-79e0-48ba-8168-64e180cb81c5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:55:52,185 [203] [DEBUG] [app] Ending request: urn:request:98b1f11b-bf60-4758-b59d-18b2a0291dbf (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:98b1f11b-bf60-4758-b59d-18b2a0291dbf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:55:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:55:52,185 [210] [DEBUG] [app] Ending request: urn:request:7478cafc-79e0-48ba-8168-64e180cb81c5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7478cafc-79e0-48ba-8168-64e180cb81c5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:55:52,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:55:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:55:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:55:52,191 [199] [DEBUG] [app] Starting request: urn:request:562ecd3e-3da5-44e4-b165-7ab474bd5552 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:55:52,191 [199] [DEBUG] [app] Ending request: urn:request:562ecd3e-3da5-44e4-b165-7ab474bd5552 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:562ecd3e-3da5-44e4-b165-7ab474bd5552', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:52,191 [198] [DEBUG] [app] Starting request: urn:request:4d2b5bf7-5397-4848-915e-b9c8c02cfb62 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:55:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:55:52,192 [198] [DEBUG] [app] Ending request: urn:request:4d2b5bf7-5397-4848-915e-b9c8c02cfb62 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4d2b5bf7-5397-4848-915e-b9c8c02cfb62', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:55:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:52,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:55:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:55:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:55:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:55:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:55:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:55:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:55:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:55:52,203 [197] [DEBUG] [app] Ending request: urn:request:5389376c-0db0-42cf-b0c3-65f6f85da95d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5389376c-0db0-42cf-b0c3-65f6f85da95d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:55:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:52,203 [200] [DEBUG] [app] Ending request: urn:request:104379df-b9b1-4279-abfc-208763487840 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:104379df-b9b1-4279-abfc-208763487840', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:55:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:55:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:55:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:55:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 05:55:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:55:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:55:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 05:55:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:55:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:02.293757+00:00 (in 9.999454 seconds) chunkcleanupworker stdout | 2023-08-08 05:55:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:02 UTC)" (scheduled at 2023-08-08 05:55:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:55:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:55:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 52, 294702), True, datetime.datetime(2023, 8, 8, 5, 55, 52, 294702), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:55:52,301 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:55:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:55:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:55:52,825 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:55:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:55:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:13.097375+00:00 (in 18.812704 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:55:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:54 UTC)" (scheduled at 2023-08-08 05:55:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:55:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:55:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:55:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:55:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:36.579841+00:00 (in 42.185675 seconds) namespacegcworker stdout | 2023-08-08 05:55:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:54 UTC)" (scheduled at 2023-08-08 05:55:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:55:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:55:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 54, 394666), True, datetime.datetime(2023, 8, 8, 5, 55, 54, 394666), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:55:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:55:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:55:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:56:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:55:54,614 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:55:54,677 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:55:54,767 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:55:54,771 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:55:54,775 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:55:54,971 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:55:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:55:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:02.178530+00:00 (in 6.806320 seconds) notificationworker stdout | 2023-08-08 05:55:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:05 UTC)" (scheduled at 2023-08-08 05:55:55.371682+00:00) notificationworker stdout | 2023-08-08 05:55:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:55:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 55, 55, 372583), True, datetime.datetime(2023, 8, 8, 5, 55, 55, 372583), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:55:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:55:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:55:55,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:55:56,115 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:55:56,630 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:55:56,819 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:55:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:55:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:19.687363+00:00 (in 22.796089 seconds) securityworker stdout | 2023-08-08 05:55:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:26 UTC)" (scheduled at 2023-08-08 05:55:56.890741+00:00) securityworker stdout | 2023-08-08 05:55:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:55:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:55:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:55:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:55:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:55:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:55:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:55:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:55:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:55:56,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:55:56,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:55:56,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:55:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:55:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:55:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:55:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:55:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:55:56,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:55:57,176 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:55:57,190 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:55:57,199 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:55:57,209 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:55:57,213 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:55:58,010 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:55:58,165 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:55:59,256 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:55:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:55:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:09.417660+00:00 (in 9.999468 seconds) gcworker stdout | 2023-08-08 05:55:59,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:09 UTC)" (scheduled at 2023-08-08 05:55:59.417660+00:00) gcworker stdout | 2023-08-08 05:55:59,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264559419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:55:59,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:55:59,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:55:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:56:00,054 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:56:00,162 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:56:00,586 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:56:00,595 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:56:00,595 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:56:00,600 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:56:00,681 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:56:01,195 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:56:01,458 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:56:01,798 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:56:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:56:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:05.371682+00:00 (in 3.192636 seconds) notificationworker stdout | 2023-08-08 05:56:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:02 UTC)" (scheduled at 2023-08-08 05:56:02.178530+00:00) notificationworker stdout | 2023-08-08 05:56:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:56:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:56:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:56:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:12.293757+00:00 (in 9.999466 seconds) chunkcleanupworker stdout | 2023-08-08 05:56:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:12 UTC)" (scheduled at 2023-08-08 05:56:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:56:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:56:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 2, 294691), True, datetime.datetime(2023, 8, 8, 5, 56, 2, 294691), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:56:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:56:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:56:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:56:02,985 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:56:02,988 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:56:02,992 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:56:02,995 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:56:02,997 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:56:03,002 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:56:03,006 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:56:03,009 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:56:03,012 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:56:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:56:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:09.793658+00:00 (in 5.605714 seconds) repositorygcworker stdout | 2023-08-08 05:56:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:04 UTC)" (scheduled at 2023-08-08 05:56:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:56:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:56:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:56:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:56:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:15.371682+00:00 (in 9.999493 seconds) notificationworker stdout | 2023-08-08 05:56:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:15 UTC)" (scheduled at 2023-08-08 05:56:05.371682+00:00) notificationworker stdout | 2023-08-08 05:56:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:56:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 5, 372541), True, datetime.datetime(2023, 8, 8, 5, 56, 5, 372541), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:56:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:56:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:56:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:07,181 [200] [DEBUG] [app] Starting request: urn:request:3cb0ad37-c954-40e0-ae8b-963ca2dc1c14 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:56:07,182 [197] [DEBUG] [app] Starting request: urn:request:87d24790-1d38-4ad8-8b89-f286e50a0a85 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:56:07,183 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:07,183 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:07,186 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:07,186 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:56:07,187 [207] [DEBUG] [app] Starting request: urn:request:dacfdf9a-1406-454f-b375-67f5bb916c3d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:56:07,187 [209] [DEBUG] [app] Starting request: urn:request:b6eb416f-0e3d-47fb-9642-838ae8a99b2b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:56:07,187 [207] [DEBUG] [app] Ending request: urn:request:dacfdf9a-1406-454f-b375-67f5bb916c3d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:dacfdf9a-1406-454f-b375-67f5bb916c3d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:56:07,188 [209] [DEBUG] [app] Ending request: urn:request:b6eb416f-0e3d-47fb-9642-838ae8a99b2b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b6eb416f-0e3d-47fb-9642-838ae8a99b2b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:56:07,188 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:56:07,188 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:56:07,188 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:56:07,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:07,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:07,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:07,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:07,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:07,193 [199] [DEBUG] [app] Starting request: urn:request:7be390dd-ed2b-4e77-ac92-c50ac927b6f0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:07,193 [200] [DEBUG] [app] Starting request: urn:request:d3eaa1d0-c520-4369-a0c5-d71b2c0a4a02 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:07,193 [199] [DEBUG] [app] Ending request: urn:request:7be390dd-ed2b-4e77-ac92-c50ac927b6f0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7be390dd-ed2b-4e77-ac92-c50ac927b6f0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:56:07,194 [200] [DEBUG] [app] Ending request: urn:request:d3eaa1d0-c520-4369-a0c5-d71b2c0a4a02 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d3eaa1d0-c520-4369-a0c5-d71b2c0a4a02', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:56:07,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:56:07,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 05:56:07,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:07,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:07,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:56:07,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:56:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:56:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:37.197412+00:00 (in 29.999358 seconds) buildlogsarchiver stdout | 2023-08-08 05:56:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:37 UTC)" (scheduled at 2023-08-08 05:56:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:56:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 56, 7, 198514), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:07,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:56:07,200 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:56:07,205 [197] [DEBUG] [app] Ending request: urn:request:87d24790-1d38-4ad8-8b89-f286e50a0a85 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:87d24790-1d38-4ad8-8b89-f286e50a0a85', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:56:07,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:07,205 [200] [DEBUG] [app] Ending request: urn:request:3cb0ad37-c954-40e0-ae8b-963ca2dc1c14 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3cb0ad37-c954-40e0-ae8b-963ca2dc1c14', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:56:07,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:07,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:56:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:56:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) buildlogsarchiver stdout | 2023-08-08 05:56:07,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:56:07,205 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:07,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:56:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:56:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) buildlogsarchiver stdout | 2023-08-08 05:56:07,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:37 UTC)" executed successfully gcworker stdout | 2023-08-08 05:56:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:56:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:19.417660+00:00 (in 9.999500 seconds) gcworker stdout | 2023-08-08 05:56:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:19 UTC)" (scheduled at 2023-08-08 05:56:09.417660+00:00) gcworker stdout | 2023-08-08 05:56:09,419 [61] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2023-08-08 05:56:09,426 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264569425, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:56:09,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:56:09,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:56:09,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:56:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:56:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:04.187429+00:00 (in 54.393248 seconds) repositorygcworker stdout | 2023-08-08 05:56:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:09 UTC)" (scheduled at 2023-08-08 05:56:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:56:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:56:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 9, 794651), True, datetime.datetime(2023, 8, 8, 5, 56, 9, 794651), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:56:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:56:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:56:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:09 UTC)" executed successfully builder stdout | 2023-08-08 05:56:10,602 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:56:10,611 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:56:10,611 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:56:10,616 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:56:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:56:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:22.293757+00:00 (in 9.999458 seconds) chunkcleanupworker stdout | 2023-08-08 05:56:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:22 UTC)" (scheduled at 2023-08-08 05:56:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:56:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:56:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 12, 294701), True, datetime.datetime(2023, 8, 8, 5, 56, 12, 294701), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:56:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:56:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:56:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:56:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:56:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:31.283850+00:00 (in 18.185907 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:56:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:13 UTC)" (scheduled at 2023-08-08 05:56:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:56:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:56:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 13, 98454), True, datetime.datetime(2023, 8, 8, 5, 56, 13, 98454), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:56:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:56:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:56:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:56:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:56:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:25.371682+00:00 (in 9.999471 seconds) notificationworker stdout | 2023-08-08 05:56:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:25 UTC)" (scheduled at 2023-08-08 05:56:15.371682+00:00) notificationworker stdout | 2023-08-08 05:56:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:56:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 15, 372769), True, datetime.datetime(2023, 8, 8, 5, 56, 15, 372769), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:56:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:56:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:56:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:56:17,278 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gunicorn-web stdout | 2023-08-08 05:56:18,395 [197] [DEBUG] [app] Starting request: urn:request:3ecc9c65-4608-4e14-87f9-9342bfd1eb22 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:18,395 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:18,395 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:18,396 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:18,403 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:18,403 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:18,403 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:18,403 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:18,403 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:18,403 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:18,403 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:18,403 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:18,406 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:18,412 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:18,413 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:18,417 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:56:18,420 [197] [DEBUG] [app] Ending request: urn:request:3ecc9c65-4608-4e14-87f9-9342bfd1eb22 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:3ecc9c65-4608-4e14-87f9-9342bfd1eb22', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:18,421 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:18 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 867 0.029) gunicorn-web stdout | 2023-08-08 05:56:18,422 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:18 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" logrotateworker stdout | 2023-08-08 05:56:18,511 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:56:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:56:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:29.417660+00:00 (in 9.999391 seconds) gcworker stdout | 2023-08-08 05:56:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:29 UTC)" (scheduled at 2023-08-08 05:56:19.417660+00:00) gcworker stdout | 2023-08-08 05:56:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264579418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:56:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:56:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:56:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:29 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:56:19,667 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} securityworker stdout | 2023-08-08 05:56:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:56:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:26.890741+00:00 (in 7.202853 seconds) securityworker stdout | 2023-08-08 05:56:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:49 UTC)" (scheduled at 2023-08-08 05:56:19.687363+00:00) securityworker stdout | 2023-08-08 05:56:19,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:56:19,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:56:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:56:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:56:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:56:19,700 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:56:19,700 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:49 UTC)" executed successfully storagereplication stdout | 2023-08-08 05:56:20,027 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:56:20,618 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:56:20,627 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:56:20,627 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:56:20,631 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:22,179 [197] [DEBUG] [app] Starting request: urn:request:0c37936a-75b0-4e97-a8c8-0428b7f2e294 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:56:22,180 [200] [DEBUG] [app] Starting request: urn:request:65e156ad-08f1-4e93-b520-103bb00ce140 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:56:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:56:22,185 [209] [DEBUG] [app] Starting request: urn:request:ab427dbd-54c4-484c-aa80-cd3e219a3cb6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:56:22,185 [206] [DEBUG] [app] Starting request: urn:request:40be3fb5-5857-4409-ac10-2fb4749cf4ec (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:56:22,185 [209] [DEBUG] [app] Ending request: urn:request:ab427dbd-54c4-484c-aa80-cd3e219a3cb6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ab427dbd-54c4-484c-aa80-cd3e219a3cb6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:56:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:56:22,186 [206] [DEBUG] [app] Ending request: urn:request:40be3fb5-5857-4409-ac10-2fb4749cf4ec (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:40be3fb5-5857-4409-ac10-2fb4749cf4ec', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:56:22,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:56:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:56:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:22,191 [199] [DEBUG] [app] Starting request: urn:request:c0b314e4-c075-4131-87ef-18abf0391a4d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:22,191 [197] [DEBUG] [app] Starting request: urn:request:9b765b94-90c6-428b-870f-9875f4ee8bdf (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:22,191 [199] [DEBUG] [app] Ending request: urn:request:c0b314e4-c075-4131-87ef-18abf0391a4d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c0b314e4-c075-4131-87ef-18abf0391a4d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:56:22,191 [197] [DEBUG] [app] Ending request: urn:request:9b765b94-90c6-428b-870f-9875f4ee8bdf (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9b765b94-90c6-428b-870f-9875f4ee8bdf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:56:22,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:56:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:56:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:56:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:56:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:56:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:56:22,202 [197] [DEBUG] [app] Ending request: urn:request:0c37936a-75b0-4e97-a8c8-0428b7f2e294 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0c37936a-75b0-4e97-a8c8-0428b7f2e294', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:56:22,202 [200] [DEBUG] [app] Ending request: urn:request:65e156ad-08f1-4e93-b520-103bb00ce140 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:65e156ad-08f1-4e93-b520-103bb00ce140', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:56:22,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:22,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:56:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:56:22,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:56:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:56:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:56:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) chunkcleanupworker stdout | 2023-08-08 05:56:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:56:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:28.874906+00:00 (in 6.580628 seconds) chunkcleanupworker stdout | 2023-08-08 05:56:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:32 UTC)" (scheduled at 2023-08-08 05:56:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:56:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:56:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 22, 294758), True, datetime.datetime(2023, 8, 8, 5, 56, 22, 294758), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:56:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:56:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:56:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:32 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:22,398 [197] [DEBUG] [app] Starting request: urn:request:e857e672-1818-4d87-9d05-74f879b7995f (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:22,398 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:22,398 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:22,399 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:22,405 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:22,406 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:22,406 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:22,406 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:22,406 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:22,406 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:22,406 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:22,406 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:22,409 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:22,415 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:22,416 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:22,420 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:56:22,423 [197] [DEBUG] [app] Ending request: urn:request:e857e672-1818-4d87-9d05-74f879b7995f (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:e857e672-1818-4d87-9d05-74f879b7995f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:22,424 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:22 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 417 0.028) gunicorn-web stdout | 2023-08-08 05:56:22,425 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:22 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:22,478 [197] [DEBUG] [app] Starting request: urn:request:dc3d90d4-85fb-4851-9f17-795aef378d1f (/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:22,479 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:22,482 [197] [DEBUG] [app] Ending request: urn:request:dc3d90d4-85fb-4851-9f17-795aef378d1f (/repository/build_test/github) {'endpoint': 'web.repository', 'request_id': 'urn:request:dc3d90d4-85fb-4851-9f17-795aef378d1f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds', 'path': '/repository/build_test/github', 'parameters': {'tab': 'builds'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:22,483 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:22 +0000] "GET /repository/build_test/github?tab=builds HTTP/1.0" 200 85192 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:22 +0000] "GET /repository/build_test/github?tab=builds HTTP/2.0" 200 13315 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.007 186 0.006) gcworker stdout | 2023-08-08 05:56:22,832 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:23 +0000] "GET /static/build/main-quay-frontend-ab93c42f539d59de0115.bundle.js HTTP/2.0" 200 719930 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.053 441 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:24 +0000] "GET /static/img/quay_favicon.png HTTP/2.0" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 81 -) blobuploadcleanupworker stdout | 2023-08-08 05:56:24,624 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:56:24,715 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:56:24,778 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:56:24,781 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:56:24,784 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} gunicorn-web stdout | 2023-08-08 05:56:24,890 [200] [DEBUG] [app] Starting request: urn:request:ce6ba8b3-f7e7-40d9-9125-7139d64faa70 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:24,890 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,890 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,891 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:24,897 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:24,897 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:24,897 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,897 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,897 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,897 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,897 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,899 [200] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 05:56:24,902 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,902 [200] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 05:56:24,903 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:56:24,905 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:24,906 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:56:24,908 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:24,909 [200] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691474184909, 0, 1, None]) gunicorn-web stdout | 2023-08-08 05:56:24,913 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:24,914 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:56:24,915 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:24,915 [197] [DEBUG] [app] Starting request: urn:request:f322fd1e-9a23-40bf-82e1-edf34e7d02d7 (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:24,916 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,916 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,916 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:24,916 [200] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691474184916, 0, 1]) gunicorn-web stdout | 2023-08-08 05:56:24,918 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,918 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,919 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:56:24,921 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,921 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,921 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,921 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:56:24,921 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,921 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,921 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,922 [200] [DEBUG] [app] Ending request: urn:request:ce6ba8b3-f7e7-40d9-9125-7139d64faa70 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:ce6ba8b3-f7e7-40d9-9125-7139d64faa70', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:24,922 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:24 +0000] "GET /api/v1/user/ HTTP/2.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.035 46 0.034) gunicorn-web stdout | 2023-08-08 05:56:24,923 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:24 +0000] "GET /api/v1/user/ HTTP/1.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:24,923 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:24,923 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:24,923 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,924 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 05:56:24,926 [197] [DEBUG] [app] Ending request: urn:request:f322fd1e-9a23-40bf-82e1-edf34e7d02d7 (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:f322fd1e-9a23-40bf-82e1-edf34e7d02d7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:24,926 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:24,927 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:24 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:24 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 47 0.013) gunicorn-web stdout | 2023-08-08 05:56:24,990 [199] [DEBUG] [app] Starting request: urn:request:a4163fa9-b350-4293-8b04-1b87bac2af01 (/api/v1/repository/build_test/github/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:24,990 [200] [DEBUG] [app] Starting request: urn:request:9c9f70ac-f034-44b6-9fd8-50bbe5789d85 (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:24,990 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,990 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,990 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,990 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,991 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:24,991 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:24,997 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:24,997 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:24,998 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:24,998 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:24,998 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,998 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,998 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:24,998 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,998 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,998 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:24,998 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,998 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,998 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:24,998 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,998 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:24,998 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,002 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:25,002 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) chunkcleanupworker stdout | 2023-08-08 05:56:25,008 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 05:56:25,009 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,009 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,009 [200] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:25,011 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,011 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,013 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [2, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,015 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,015 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [2, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:56:25,016 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,016 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,017 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [2, None, 1691474185017, False, 101, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,018 [200] [DEBUG] [app] Ending request: urn:request:9c9f70ac-f034-44b6-9fd8-50bbe5789d85 (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:9c9f70ac-f034-44b6-9fd8-50bbe5789d85', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:25,019 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/2.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 88 0.030) gunicorn-web stdout | 2023-08-08 05:56:25,019 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:25,020 [199] [DEBUG] [app] Ending request: urn:request:a4163fa9-b350-4293-8b04-1b87bac2af01 (/api/v1/repository/build_test/github/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:a4163fa9-b350-4293-8b04-1b87bac2af01', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/github/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:25,021 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.033 92 0.032) gunicorn-web stdout | 2023-08-08 05:56:25,021 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:25,303 [197] [DEBUG] [app] Starting request: urn:request:84edc213-7445-4aa4-97bb-468997a7c543 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:25,303 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,303 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,304 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,304 [200] [DEBUG] [app] Starting request: urn:request:682b0320-5386-41a1-9cea-cf4f2c0e2dcb (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:25,305 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,305 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,305 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,310 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:25,310 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:25,310 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,310 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:56:25,311 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,311 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,311 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,311 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,311 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:25,311 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:25,311 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,311 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:56:25,311 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,311 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,311 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,311 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,312 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 8, 25, 312120), datetime.datetime(2023, 8, 8, 5, 56, 25, 312173)]) gunicorn-web stdout | 2023-08-08 05:56:25,312 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 05:56:25,314 [200] [DEBUG] [app] Ending request: urn:request:682b0320-5386-41a1-9cea-cf4f2c0e2dcb (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:682b0320-5386-41a1-9cea-cf4f2c0e2dcb', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:25,315 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 62 0.012) gunicorn-web stdout | 2023-08-08 05:56:25,315 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:25,317 [197] [DEBUG] [app] Ending request: urn:request:84edc213-7445-4aa4-97bb-468997a7c543 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:84edc213-7445-4aa4-97bb-468997a7c543', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:25,318 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 400 0.016) gunicorn-web stdout | 2023-08-08 05:56:25,318 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 05:56:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:56:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:35.371682+00:00 (in 9.999400 seconds) notificationworker stdout | 2023-08-08 05:56:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:35 UTC)" (scheduled at 2023-08-08 05:56:25.371682+00:00) notificationworker stdout | 2023-08-08 05:56:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:56:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 25, 372656), True, datetime.datetime(2023, 8, 8, 5, 56, 25, 372656), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:56:25,379 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:56:25,379 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:56:25,379 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:56:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:56:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:47.398551+00:00 (in 21.883570 seconds) exportactionlogsworker stdout | 2023-08-08 05:56:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:25 UTC)" (scheduled at 2023-08-08 05:56:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:56:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:56:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:25,588 [197] [DEBUG] [app] Starting request: urn:request:528f8114-5a29-47b9-b2db-accc6b8a5ad3 (/api/v1/repository/build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:25,588 [200] [DEBUG] [app] Starting request: urn:request:f670165f-3664-4d3d-b227-43efdbfed114 (/api/v1/repository/build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:25,588 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,588 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,588 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,588 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,589 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,589 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,596 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:25,596 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:25,596 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:25,596 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:25,596 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,596 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,596 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:25,596 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:25,596 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,596 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,596 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,596 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,596 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,596 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,596 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,596 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,598 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:25,599 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:25,605 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,605 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,606 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,606 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,609 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'github']) gunicorn-web stdout | 2023-08-08 05:56:25,609 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'github']) gunicorn-web stdout | 2023-08-08 05:56:25,611 [200] [DEBUG] [app] Ending request: urn:request:f670165f-3664-4d3d-b227-43efdbfed114 (/api/v1/repository/build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:f670165f-3664-4d3d-b227-43efdbfed114', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/', 'path': '/api/v1/repository/build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:25,611 [197] [DEBUG] [app] Ending request: urn:request:528f8114-5a29-47b9-b2db-accc6b8a5ad3 (/api/v1/repository/build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:528f8114-5a29-47b9-b2db-accc6b8a5ad3', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/', 'path': '/api/v1/repository/build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:25,612 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:25,612 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:25,612 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 415 0.025) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 413 0.026) gunicorn-web stdout | 2023-08-08 05:56:25,612 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:25,769 [197] [DEBUG] [app] Starting request: urn:request:8e138e63-e72a-4578-bc6a-cfa98bd4a432 (/repository/build_test/github/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:25,770 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,777 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,779 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,782 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['github', 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,788 [197] [DEBUG] [app] Ending request: urn:request:8e138e63-e72a-4578-bc6a-cfa98bd4a432 (/repository/build_test/github/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:8e138e63-e72a-4578-bc6a-cfa98bd4a432', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f', 'path': '/repository/build_test/github/status', 'parameters': {'token': '245e22c3-ba4f-4e53-9a58-fce8fb4eb90f'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:25,788 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:25 +0000] "GET /repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.022 90 0.022) gunicorn-web stdout | 2023-08-08 05:56:25,789 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:25 +0000] "GET /repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:25,929 [200] [DEBUG] [app] Starting request: urn:request:62f64735-413a-4e0c-9581-bac92469a45b (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:25,929 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,929 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,930 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,936 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:25,936 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:25,936 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,936 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:25,936 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:25,937 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,937 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,937 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,939 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:25,945 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:25,946 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:25,950 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:56:25,954 [200] [DEBUG] [app] Ending request: urn:request:62f64735-413a-4e0c-9581-bac92469a45b (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:62f64735-413a-4e0c-9581-bac92469a45b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:25,954 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 73 0.028) gunicorn-web stdout | 2023-08-08 05:56:25,955 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:25 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" teamsyncworker stdout | 2023-08-08 05:56:26,127 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 05:56:26,278 [197] [DEBUG] [app] Starting request: urn:request:26ef3bb2-aaf2-4ac6-9f08-7edc1cc65856 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:26,278 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:26,278 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:26,279 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:26,285 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:26,286 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:26,286 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:26,286 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:26,286 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:26,286 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:26,286 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:26,286 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:26,288 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:26,295 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:26,296 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:26,299 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 10]) gunicorn-web stdout | 2023-08-08 05:56:26,303 [197] [DEBUG] [app] Ending request: urn:request:26ef3bb2-aaf2-4ac6-9f08-7edc1cc65856 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:26ef3bb2-aaf2-4ac6-9f08-7edc1cc65856', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=10', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:26,304 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:26 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 73 0.028) gunicorn-web stdout | 2023-08-08 05:56:26,305 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:26 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" queuecleanupworker stdout | 2023-08-08 05:56:26,649 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:56:26,826 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:56:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:56:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:49.687363+00:00 (in 22.795857 seconds) securityworker stdout | 2023-08-08 05:56:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:56 UTC)" (scheduled at 2023-08-08 05:56:26.890741+00:00) securityworker stdout | 2023-08-08 05:56:26,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:56:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:56:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:56:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:56:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:56:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:56:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:56:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:56:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:56:26,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:56:26,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:56:26,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:56:26,902 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:56:26,902 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:56:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:56:26,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:56:26,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:56:26,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:56:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:27,185 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:56:27,202 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:56:27,209 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:56:27,221 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:56:27,224 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:56:28,029 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:56:28,202 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:56:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:56:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:32.293757+00:00 (in 3.418349 seconds) chunkcleanupworker stdout | 2023-08-08 05:56:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:28 UTC)" (scheduled at 2023-08-08 05:56:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:56:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:56:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:56:29,287 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:56:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:56:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:39.417660+00:00 (in 9.999485 seconds) gcworker stdout | 2023-08-08 05:56:29,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:39 UTC)" (scheduled at 2023-08-08 05:56:29.417660+00:00) gcworker stdout | 2023-08-08 05:56:29,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264589419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:56:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:56:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:56:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:39 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:29,989 [199] [DEBUG] [app] Starting request: urn:request:e03ccc80-be41-469a-8366-fbf467cd18e3 (/oauth2/github/callback/trigger/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:29,989 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:29,989 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:29,990 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:29,997 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:29,997 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:29,997 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:29,997 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:29,997 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:29,997 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:29,999 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:30,006 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:30,009 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443 exportactionlogsworker stdout | 2023-08-08 05:56:30,077 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} gunicorn-web stdout | 2023-08-08 05:56:30,148 [199] [DEBUG] [urllib3.connectionpool] https://github.com:443 "POST /login/oauth/access_token?code=34d5480b43f61464a869&grant_type=authorization_code&redirect_uri=https%3A%2F%2Fquayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com%2Foauth2%2Fgithub%2Fcallback&client_id=2ddf9640aa6b5e341a7d&client_secret=3764a2acebe77c5406143cb606e79683e994ae38 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:30,149 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:30,152 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:30,153 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "buildtriggerservice" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['github', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:30,154 [199] [DEBUG] [peewee] ('INSERT INTO "repositorybuildtrigger" ("uuid", "service_id", "repository_id", "connected_user_id", "secure_auth_token", "fully_migrated", "config", "pull_robot_id", "enabled", "disabled_datetime", "successive_failure_count", "successive_internal_error_count") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repositorybuildtrigger"."id"', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 2, 1, 'v0$$5V29IBxCwbRzoUWHFpD1NiZkzW/p9zx8uAKuMCjrK5Q2K0AUwYSRNHE8IvOu+AlW1LluSDoW9j4944XcROydzrkDEyrb', False, '{}', None, True, datetime.datetime(2023, 8, 8, 5, 56, 30, 154069), 0, 0]) servicekey stdout | 2023-08-08 05:56:30,169 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} gunicorn-web stdout | 2023-08-08 05:56:30,175 [199] [DEBUG] [endpoints.githubtrigger] Redirecting to full url: /repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b gunicorn-web stdout | 2023-08-08 05:56:30,175 [199] [DEBUG] [app] Ending request: urn:request:e03ccc80-be41-469a-8366-fbf467cd18e3 (/oauth2/github/callback/trigger/build_test/github) {'endpoint': 'callback.attach_github_build_trigger', 'request_id': 'urn:request:e03ccc80-be41-469a-8366-fbf467cd18e3', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/oauth2/github/callback/trigger/build_test/github?code=34d5480b43f61464a869', 'path': '/oauth2/github/callback/trigger/build_test/github', 'parameters': {'code': '34d5480b43f61464a869'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:30,176 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:30 +0000] "GET /oauth2/github/callback/trigger/build_test/github?code=34d5480b43f61464a869 HTTP/2.0" 302 355 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.189 562 0.189) gunicorn-web stdout | 2023-08-08 05:56:30,177 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:30 +0000] "GET /oauth2/github/callback/trigger/build_test/github?code=34d5480b43f61464a869 HTTP/1.0" 302 355 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:30,495 [197] [DEBUG] [app] Starting request: urn:request:7baf7d63-5aaa-45f1-9f3e-73264f514dd5 (/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:30,497 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:30,499 [197] [DEBUG] [app] Ending request: urn:request:7baf7d63-5aaa-45f1-9f3e-73264f514dd5 (/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b) {'endpoint': 'web.buildtrigger', 'request_id': 'urn:request:7baf7d63-5aaa-45f1-9f3e-73264f514dd5', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 'path': '/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:30,500 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:30 +0000] "GET /repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b HTTP/1.0" 200 85192 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:30 +0000] "GET /repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b HTTP/2.0" 200 13315 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.007 432 0.006) builder stdout | 2023-08-08 05:56:30,641 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:56:30,650 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:56:30,650 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:56:30,654 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:56:30,688 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:56:31,223 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2023-08-08 05:56:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:56:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:54.284093+00:00 (in 22.999681 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:56:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:01:31 UTC)" (scheduled at 2023-08-08 05:56:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:56:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 56, 31, 284903), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:56:31,292 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 56, 31, 284903), True, datetime.datetime(2023, 8, 8, 5, 56, 31, 284903), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:56:31,294 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 56, 31, 284903), True, datetime.datetime(2023, 8, 8, 5, 56, 31, 284903), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 5, 56, 31, 284903), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 05:56:31,295 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:56:31,295 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:01:31 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:31 +0000] "GET /static/build/489e5e00e1a9a80f88d55281894a87ff.woff HTTP/2.0" 200 7280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 264 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:31 +0000] "GET /static/img/quay_favicon.png HTTP/2.0" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 54 -) gunicorn-web stdout | 2023-08-08 05:56:31,365 [197] [DEBUG] [app] Starting request: urn:request:17659424-bff6-484e-bb8c-ac78a7bc0301 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:31,365 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,365 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,366 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,372 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:31,372 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:31,373 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,373 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,373 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,373 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,373 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,374 [197] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 05:56:31,377 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,377 [197] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 05:56:31,379 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 05:56:31,380 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,381 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:56:31,383 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,384 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691474191383, 0, 1, None]) gunicorn-web stdout | 2023-08-08 05:56:31,388 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,389 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 05:56:31,390 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,392 [197] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691474191391, 0, 1]) gunicorn-web stdout | 2023-08-08 05:56:31,393 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,393 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,393 [199] [DEBUG] [app] Starting request: urn:request:ddc19cd3-1948-4e12-a169-343a9e67acdf (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:31,394 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,394 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,394 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:56:31,394 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,396 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,396 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,396 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,396 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:56:31,396 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,396 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,397 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,397 [197] [DEBUG] [app] Ending request: urn:request:17659424-bff6-484e-bb8c-ac78a7bc0301 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:17659424-bff6-484e-bb8c-ac78a7bc0301', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:31,398 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:31,398 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/user/ HTTP/1.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/user/ HTTP/2.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.035 52 0.035) gunicorn-web stdout | 2023-08-08 05:56:31,402 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:31,402 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:31,402 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,402 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 05:56:31,404 [199] [DEBUG] [app] Ending request: urn:request:ddc19cd3-1948-4e12-a169-343a9e67acdf (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:ddc19cd3-1948-4e12-a169-343a9e67acdf', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:31,405 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 47 0.012) gunicorn-web stdout | 2023-08-08 05:56:31,405 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:56:31,474 [199] [DEBUG] [app] Starting request: urn:request:da601560-af2f-48d1-8cc6-02cb0ae444ff (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:31,474 [198] [DEBUG] [app] Starting request: urn:request:6f628ea2-ee0e-4929-86af-2468c91a63db (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:31,474 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,474 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,474 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,474 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,475 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,475 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) namespacegcworker stdout | 2023-08-08 05:56:31,476 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-web stdout | 2023-08-08 05:56:31,481 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:31,481 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:31,481 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,481 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:31,481 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,481 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:31,481 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,481 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,481 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,481 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:31,481 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,482 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:31,482 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,482 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,482 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,482 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,483 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:31,484 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:31,489 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,490 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,490 [199] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:31,490 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,491 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,492 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [2, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,494 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,494 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [2, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:56:31,495 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,495 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,496 [199] [DEBUG] [app] Ending request: urn:request:da601560-af2f-48d1-8cc6-02cb0ae444ff (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:da601560-af2f-48d1-8cc6-02cb0ae444ff', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github?includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:31,496 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:31,497 [198] [DEBUG] [app] Ending request: urn:request:6f628ea2-ee0e-4929-86af-2468c91a63db (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:6f628ea2-ee0e-4929-86af-2468c91a63db', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:31,497 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/1.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/2.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.024 75 0.024) gunicorn-web stdout | 2023-08-08 05:56:31,497 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b HTTP/2.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 93 0.025) gunicorn-web stdout | 2023-08-08 05:56:31,498 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b HTTP/1.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" expiredappspecifictokenworker stdout | 2023-08-08 05:56:31,813 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-web stdout | 2023-08-08 05:56:31,857 [198] [DEBUG] [app] Starting request: urn:request:fbb9bea9-1707-414c-a379-e8bd46fe2a79 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:31,857 [200] [DEBUG] [app] Starting request: urn:request:183ec280-d8c3-4625-a16a-6afa53fa7c18 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:31,857 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,857 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,857 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,857 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,858 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,858 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,865 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:31,865 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:31,865 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:31,865 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,865 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:31,865 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,865 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:56:31,865 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,865 [198] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 05:56:31,865 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,865 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,865 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,865 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,865 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,865 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,865 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,866 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 8, 31, 865783), datetime.datetime(2023, 8, 8, 5, 56, 31, 865837)]) gunicorn-web stdout | 2023-08-08 05:56:31,867 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 05:56:31,867 [198] [DEBUG] [app] Ending request: urn:request:fbb9bea9-1707-414c-a379-e8bd46fe2a79 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:fbb9bea9-1707-414c-a379-e8bd46fe2a79', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:31,868 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:31,868 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 62 0.013) gunicorn-web stdout | 2023-08-08 05:56:31,871 [200] [DEBUG] [app] Ending request: urn:request:183ec280-d8c3-4625-a16a-6afa53fa7c18 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:183ec280-d8c3-4625-a16a-6afa53fa7c18', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:31,872 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:31,872 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:31 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 399 0.016) gunicorn-web stdout | 2023-08-08 05:56:31,897 [199] [DEBUG] [app] Starting request: urn:request:35f54997-f739-43fc-818e-9a51816a1e49 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/namespaces) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:31,897 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,897 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,898 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,904 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:31,904 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:31,904 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,905 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:31,905 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,905 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,905 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,905 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,907 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:31,913 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:31,914 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,917 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,920 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:31,921 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:31,922 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:56:32,053 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:32,054 [199] [DEBUG] [github.Requester] GET https://api.github.com/user {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:32 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"0f517dbf9652741aa942bae201014bb523a1f77d872dd5df908cee138ee353fb"', 'last-modified': 'Fri, 04 Aug 2023 13:02:06 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4972', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '28', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F18:82E4:E5EB2C:1D45D78:64D1D90F'} {"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false,"name":"Hu Weihua","company":null,"blog":"","location":null,"email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":15,"public_gists":0,"followers":5,"following":0,"created_at":"2014-04-22T08:41:09Z","updated_at":"2023-08-04T13:02:06Z"} gunicorn-web stdout | 2023-08-08 05:56:32,129 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user/orgs HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:32,130 [199] [DEBUG] [github.Requester] GET https://api.github.com/user/orgs {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:32 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"ad2699e1b1fb69bf9495f33b8820ff4ed75bc76ed9f9d4537557ad922f11dd25"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:org, read:org, repo, user, write:org', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4971', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '29', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F18:82E4:E5EB48:1D45DAF:64D1D910'} [{"login":"finley","id":422903,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQyMjkwMw==","url":"https://api.github.com/orgs/finley","repos_url":"https://api.github.com/orgs/finley/repos","events_url":"https://api.github.com/orgs/finley/events","hooks_url":"https://api.github.com/orgs/finley/hooks","issues_url":"https://api.github.com/orgs/finley/issues","members_url":"https://api.github.com/orgs/finley/members{/member}","public_members_url":"https://api.github.com/orgs/finley/public_members{/member}","avatar_url":"https://avatars.githubusercontent.com/u/422903?v=4","description":null},{"login":"xcat2","id":10124414,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwMTI0NDE0","url":"https://api.github.com/orgs/xcat2","repos_url":"https://api.github.com/orgs/xcat2/repos","events_url":"https://api.github.com/orgs/xcat2/events","hooks_url":"https://api.github.com/orgs/xcat2/hooks","issues_url":"https://api.github.com/orgs/xcat2/issues","members_url":"https://api.github.com/orgs/xcat2/members{/member}","public_members_url":"https://api.github.com/orgs/xcat2/public_members{/member}","avatar_url":"https://avatars.githubusercontent.com/u/10124414?v=4","description":" xCAT offers complete management for HPC clusters, RenderFarms, Grids, WebFarms, Online Gaming Infrastructure, Clouds, Datacenters"}] gunicorn-web stdout | 2023-08-08 05:56:32,133 [199] [DEBUG] [app] Ending request: urn:request:35f54997-f739-43fc-818e-9a51816a1e49 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/namespaces) {'endpoint': 'api.buildtriggersourcenamespaces', 'request_id': 'urn:request:35f54997-f739-43fc-818e-9a51816a1e49', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/namespaces', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/namespaces', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:32,134 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:32,134 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:32 +0000] "GET /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/namespaces HTTP/1.0" 200 492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:32 +0000] "GET /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/namespaces HTTP/2.0" 200 492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.238 100 0.239) chunkcleanupworker stdout | 2023-08-08 05:56:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:56:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:42.293757+00:00 (in 9.999444 seconds) chunkcleanupworker stdout | 2023-08-08 05:56:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:42 UTC)" (scheduled at 2023-08-08 05:56:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:56:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:56:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 32, 294860), True, datetime.datetime(2023, 8, 8, 5, 56, 32, 294860), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:56:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:56:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:56:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:56:32,998 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:56:33,001 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:56:33,004 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:56:33,007 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:56:33,009 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:56:33,012 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:56:33,016 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:56:33,019 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:56:33,021 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-web stdout | 2023-08-08 05:56:34,375 [200] [DEBUG] [app] Starting request: urn:request:cde571e7-40a8-4502-81ba-8ba658284fd6 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/sources) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:34,375 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:34,375 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:34,376 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:34,384 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:34,384 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:34,384 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:34,384 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:34,384 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:34,385 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:34,385 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:34,385 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:34,388 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:34,395 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:34,396 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:34,399 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:34,402 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:34,405 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:34,406 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:34,409 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:56:34,518 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:34,519 [200] [DEBUG] [github.Requester] GET https://api.github.com/user {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:34 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"0f517dbf9652741aa942bae201014bb523a1f77d872dd5df908cee138ee353fb"', 'last-modified': 'Fri, 04 Aug 2023 13:02:06 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4970', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '30', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F1E:82E4:E5EE2E:1D463B8:64D1D912'} {"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false,"name":"Hu Weihua","company":null,"blog":"","location":null,"email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":15,"public_gists":0,"followers":5,"following":0,"created_at":"2014-04-22T08:41:09Z","updated_at":"2023-08-04T13:02:06Z"} gunicorn-web stdout | 2023-08-08 05:56:34,823 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user/repos?type=owner&sort=updated HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:34,824 [200] [DEBUG] [github.Requester] GET https://api.github.com/user/repos?type=owner&sort=updated {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:34 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"ef24073681721206f633dba4219c70a892feb17e99827c8e05be7afe62443d97"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4969', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '31', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F1E:82E4:E5EE44:1D463EC:64D1D912'} [{"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":473030298,"node_id":"R_kgDOHDHemg","name":"quay_build_test_repo2","full_name":"hu-weihua/quay_build_test_repo2","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test_repo2","description":"quay_build_test_repo2","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/deployments","created_at":"2022-03-23T04:00:22Z","updated_at":"2022-03-23T04:01:50Z","pushed_at":"2022-03-23T04:01:47Z","git_url":"git://github.com/hu-weihua/quay_build_test_repo2.git","ssh_url":"git@github.com:hu-weihua/quay_build_test_repo2.git","clone_url":"https://github.com/hu-weihua/quay_build_test_repo2.git","svn_url":"https://github.com/hu-weihua/quay_build_test_repo2","homepage":null,"size":1,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":473864393,"node_id":"R_kgDOHD6YyQ","name":"node-app","full_name":"hu-weihua/node-app","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/node-app","description":null,"fork":true,"url":"https://api.github.com/repos/hu-weihua/node-app","forks_url":"https://api.github.com/repos/hu-weihua/node-app/forks","keys_url":"https://api.github.com/repos/hu-weihua/node-app/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/node-app/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/node-app/teams","hooks_url":"https://api.github.com/repos/hu-weihua/node-app/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/node-app/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/node-app/events","assignees_url":"https://api.github.com/repos/hu-weihua/node-app/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/node-app/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/node-app/tags","blobs_url":"https://api.github.com/repos/hu-weihua/node-app/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/node-app/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/node-app/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/node-app/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/node-app/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/node-app/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/node-app/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/node-app/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/node-app/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/node-app/subscription","commits_url":"https://api.github.com/repos/hu-weihua/node-app/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/node-app/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/node-app/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/node-app/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/node-app/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/node-app/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/node-app/merges","archive_url":"https://api.github.com/repos/hu-weihua/node-app/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/node-app/downloads","issues_url":"https://api.github.com/repos/hu-weihua/node-app/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/node-app/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/node-app/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/node-app/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/node-app/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/node-app/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/node-app/deployments","created_at":"2022-03-25T04:10:15Z","updated_at":"2021-12-30T22:39:01Z","pushed_at":"2022-03-11T17:49:16Z","git_url":"git://github.com/hu-weihua/node-app.git","ssh_url":"git@github.com:hu-weihua/node-app.git","clone_url":"https://github.com/hu-weihua/node-app.git","svn_url":"https://github.com/hu-weihua/node-app","homepage":null,"size":82,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":140396693,"node_id":"MDEwOlJlcG9zaXRvcnkxNDAzOTY2OTM=","name":"personal_repo","full_name":"hu-weihua/personal_repo","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/personal_repo","description":"This is repository of hwh","fork":false,"url":"https://api.github.com/repos/hu-weihua/personal_repo","forks_url":"https://api.github.com/repos/hu-weihua/personal_repo/forks","keys_url":"https://api.github.com/repos/hu-weihua/personal_repo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/personal_repo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/personal_repo/teams","hooks_url":"https://api.github.com/repos/hu-weihua/personal_repo/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/personal_repo/events","assignees_url":"https://api.github.com/repos/hu-weihua/personal_repo/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/personal_repo/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/personal_repo/tags","blobs_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/personal_repo/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/personal_repo/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/personal_repo/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/personal_repo/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/personal_repo/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/personal_repo/subscription","commits_url":"https://api.github.com/repos/hu-weihua/personal_repo/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/personal_repo/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/personal_repo/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/personal_repo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/personal_repo/merges","archive_url":"https://api.github.com/repos/hu-weihua/personal_repo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/personal_repo/downloads","issues_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/personal_repo/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/personal_repo/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/personal_repo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/personal_repo/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/personal_repo/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/personal_repo/deployments","created_at":"2018-07-10T07:49:54Z","updated_at":"2021-11-04T03:05:14Z","pushed_at":"2021-11-04T03:05:11Z","git_url":"git://github.com/hu-weihua/personal_repo.git","ssh_url":"git@github.com:hu-weihua/personal_repo.git","clone_url":"https://github.com/hu-weihua/personal_repo.git","svn_url":"https://github.com/hu-weihua/personal_repo","homepage":null,"size":45089,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":411522280,"node_id":"R_kgDOGIdU6A","name":"quay_tools","full_name":"hu-weihua/quay_tools","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_tools","description":"scripts for quay ","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_tools","forks_url":"https://api.github.com/repos/hu-weihua/quay_tools/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_tools/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_tools/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_tools/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_tools/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_tools/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_tools/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_tools/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_tools/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_tools/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_tools/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_tools/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_tools/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_tools/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_tools/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_tools/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_tools/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_tools/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_tools/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_tools/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_tools/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_tools/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_tools/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_tools/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_tools/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_tools/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_tools/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_tools/deployments","created_at":"2021-09-29T03:52:46Z","updated_at":"2021-10-14T04:53:01Z","pushed_at":"2021-10-14T04:52:58Z","git_url":"git://github.com/hu-weihua/quay_tools.git","ssh_url":"git@github.com:hu-weihua/quay_tools.git","clone_url":"https://github.com/hu-weihua/quay_tools.git","svn_url":"https://github.com/hu-weihua/quay_tools","homepage":null,"size":5,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":409051481,"node_id":"R_kgDOGGGhWQ","name":"quay_fedaro_laptop_setup","full_name":"hu-weihua/quay_fedaro_laptop_setup","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup","forks_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/deployments","created_at":"2021-09-22T03:25:19Z","updated_at":"2021-09-22T03:32:29Z","pushed_at":"2021-09-22T03:32:26Z","git_url":"git://github.com/hu-weihua/quay_fedaro_laptop_setup.git","ssh_url":"git@github.com:hu-weihua/quay_fedaro_laptop_setup.git","clone_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup.git","svn_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup","homepage":null,"size":2,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":251479333,"node_id":"MDEwOlJlcG9zaXRvcnkyNTE0NzkzMzM=","name":"cam_tools","full_name":"hu-weihua/cam_tools","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/cam_tools","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/cam_tools","forks_url":"https://api.github.com/repos/hu-weihua/cam_tools/forks","keys_url":"https://api.github.com/repos/hu-weihua/cam_tools/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/cam_tools/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/cam_tools/teams","hooks_url":"https://api.github.com/repos/hu-weihua/cam_tools/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/cam_tools/events","assignees_url":"https://api.github.com/repos/hu-weihua/cam_tools/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/cam_tools/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/cam_tools/tags","blobs_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/cam_tools/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/cam_tools/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/cam_tools/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/cam_tools/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/cam_tools/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/cam_tools/subscription","commits_url":"https://api.github.com/repos/hu-weihua/cam_tools/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/cam_tools/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/cam_tools/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/cam_tools/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/cam_tools/merges","archive_url":"https://api.github.com/repos/hu-weihua/cam_tools/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/cam_tools/downloads","issues_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/cam_tools/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/cam_tools/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/cam_tools/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/cam_tools/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/cam_tools/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/cam_tools/deployments","created_at":"2020-03-31T02:22:26Z","updated_at":"2021-07-12T10:24:29Z","pushed_at":"2021-07-12T10:24:26Z","git_url":"git://github.com/hu-weihua/cam_tools.git","ssh_url":"git@github.com:hu-weihua/cam_tools.git","clone_url":"https://github.com/hu-weihua/cam_tools.git","svn_url":"https://github.com/hu-weihua/cam_tools","homepage":null,"size":151,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":265992712,"node_id":"MDEwOlJlcG9zaXRvcnkyNjU5OTI3MTI=","name":"englist_study","full_name":"hu-weihua/englist_study","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/englist_study","description":"this is repo for english study","fork":false,"url":"https://api.github.com/repos/hu-weihua/englist_study","forks_url":"https://api.github.com/repos/hu-weihua/englist_study/forks","keys_url":"https://api.github.com/repos/hu-weihua/englist_study/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/englist_study/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/englist_study/teams","hooks_url":"https://api.github.com/repos/hu-weihua/englist_study/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/englist_study/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/englist_study/events","assignees_url":"https://api.github.com/repos/hu-weihua/englist_study/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/englist_study/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/englist_study/tags","blobs_url":"https://api.github.com/repos/hu-weihua/englist_study/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/englist_study/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/englist_study/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/englist_study/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/englist_study/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/englist_study/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/englist_study/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/englist_study/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/englist_study/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/englist_study/subscription","commits_url":"https://api.github.com/repos/hu-weihua/englist_study/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/englist_study/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/englist_study/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/englist_study/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/englist_study/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/englist_study/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/englist_study/merges","archive_url":"https://api.github.com/repos/hu-weihua/englist_study/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/englist_study/downloads","issues_url":"https://api.github.com/repos/hu-weihua/englist_study/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/englist_study/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/englist_study/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/englist_study/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/englist_study/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/englist_study/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/englist_study/deployments","created_at":"2020-05-22T01:45:08Z","updated_at":"2021-05-01T00:58:19Z","pushed_at":"2021-05-01T00:58:17Z","git_url":"git://github.com/hu-weihua/englist_study.git","ssh_url":"git@github.com:hu-weihua/englist_study.git","clone_url":"https://github.com/hu-weihua/englist_study.git","svn_url":"https://github.com/hu-weihua/englist_study","homepage":null,"size":82,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":308238023,"node_id":"MDEwOlJlcG9zaXRvcnkzMDgyMzgwMjM=","name":"rh-do380-ge","full_name":"hu-weihua/rh-do380-ge","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/rh-do380-ge","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/rh-do380-ge","forks_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/forks","keys_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/teams","hooks_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/events","assignees_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/tags","blobs_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/subscription","commits_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/merges","archive_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/downloads","issues_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/deployments","created_at":"2020-10-29T06:30:04Z","updated_at":"2020-11-06T07:14:37Z","pushed_at":"2020-11-06T07:14:35Z","git_url":"git://github.com/hu-weihua/rh-do380-ge.git","ssh_url":"git@github.com:hu-weihua/rh-do380-ge.git","clone_url":"https://github.com/hu-weihua/rh-do380-ge.git","svn_url":"https://github.com/hu-weihua/rh-do380-ge","homepage":null,"size":8389,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":309275165,"node_id":"MDEwOlJlcG9zaXRvcnkzMDkyNzUxNjU=","name":"baisc_test_data","full_name":"hu-weihua/baisc_test_data","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/baisc_test_data","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/baisc_test_data","forks_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/forks","keys_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/teams","hooks_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/events","assignees_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/tags","blobs_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/subscription","commits_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/merges","archive_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/downloads","issues_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/deployments","created_at":"2020-11-02T06:12:55Z","updated_at":"2020-11-02T07:45:40Z","pushed_at":"2020-11-02T07:45:38Z","git_url":"git://github.com/hu-weihua/baisc_test_data.git","ssh_url":"git@github.com:hu-weihua/baisc_test_data.git","clone_url":"https://github.com/hu-weihua/baisc_test_data.git","svn_url":"https://github.com/hu-weihua/baisc_test_data","homepage":null,"size":7,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":308519553,"node_id":"MDEwOlJlcG9zaXRvcnkzMDg1MTk1NTM=","name":"gitops-gitops","full_name":"hu-weihua/gitops-gitops","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/gitops-gitops","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/gitops-gitops","forks_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/forks","keys_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/teams","hooks_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/events","assignees_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/tags","blobs_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/subscription","commits_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/merges","archive_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/downloads","issues_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/deployments","created_at":"2020-10-30T03:53:46Z","updated_at":"2020-10-30T04:27:19Z","pushed_at":"2020-10-30T04:27:16Z","git_url":"git://github.com/hu-weihua/gitops-gitops.git","ssh_url":"git@github.com:hu-weihua/gitops-gitops.git","clone_url":"https://github.com/hu-weihua/gitops-gitops.git","svn_url":"https://github.com/hu-weihua/gitops-gitops","homepage":null,"size":4,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":279471706,"node_id":"MDEwOlJlcG9zaXRvcnkyNzk0NzE3MDY=","name":"DO180-apps","full_name":"hu-weihua/DO180-apps","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/DO180-apps","description":"DO180 Repository for Sample Applications","fork":true,"url":"https://api.github.com/repos/hu-weihua/DO180-apps","forks_url":"https://api.github.com/repos/hu-weihua/DO180-apps/forks","keys_url":"https://api.github.com/repos/hu-weihua/DO180-apps/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/DO180-apps/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/DO180-apps/teams","hooks_url":"https://api.github.com/repos/hu-weihua/DO180-apps/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/DO180-apps/events","assignees_url":"https://api.github.com/repos/hu-weihua/DO180-apps/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/DO180-apps/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/DO180-apps/tags","blobs_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/DO180-apps/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/DO180-apps/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/DO180-apps/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/DO180-apps/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/DO180-apps/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/DO180-apps/subscription","commits_url":"https://api.github.com/repos/hu-weihua/DO180-apps/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/DO180-apps/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/DO180-apps/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/DO180-apps/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/DO180-apps/merges","archive_url":"https://api.github.com/repos/hu-weihua/DO180-apps/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/DO180-apps/downloads","issues_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/DO180-apps/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/DO180-apps/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/DO180-apps/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/DO180-apps/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/DO180-apps/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/DO180-apps/deployments","created_at":"2020-07-14T03:29:50Z","updated_at":"2020-07-14T03:29:51Z","pushed_at":"2020-07-17T08:41:06Z","git_url":"git://github.com/hu-weihua/DO180-apps.git","ssh_url":"git@github.com:hu-weihua/DO180-apps.git","clone_url":"https://github.com/hu-weihua/DO180-apps.git","svn_url":"https://github.com/hu-weihua/DO180-apps","homepage":null,"size":230,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":238137702,"node_id":"MDEwOlJlcG9zaXRvcnkyMzgxMzc3MDI=","name":"mig-e2e","full_name":"hu-weihua/mig-e2e","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/mig-e2e","description":"End to end tests for OCP Migration","fork":true,"url":"https://ap gunicorn-web stdout | i.github.com/repos/hu-weihua/mig-e2e","forks_url":"https://api.github.com/repos/hu-weihua/mig-e2e/forks","keys_url":"https://api.github.com/repos/hu-weihua/mig-e2e/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/mig-e2e/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/mig-e2e/teams","hooks_url":"https://api.github.com/repos/hu-weihua/mig-e2e/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/mig-e2e/events","assignees_url":"https://api.github.com/repos/hu-weihua/mig-e2e/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/mig-e2e/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/mig-e2e/tags","blobs_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/mig-e2e/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/mig-e2e/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/mig-e2e/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/mig-e2e/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/mig-e2e/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/mig-e2e/subscription","commits_url":"https://api.github.com/repos/hu-weihua/mig-e2e/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/mig-e2e/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/mig-e2e/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/mig-e2e/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/mig-e2e/merges","archive_url":"https://api.github.com/repos/hu-weihua/mig-e2e/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/mig-e2e/downloads","issues_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/mig-e2e/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/mig-e2e/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/mig-e2e/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/mig-e2e/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/mig-e2e/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/mig-e2e/deployments","created_at":"2020-02-04T06:21:19Z","updated_at":"2020-02-25T17:18:34Z","pushed_at":"2020-04-28T01:45:39Z","git_url":"git://github.com/hu-weihua/mig-e2e.git","ssh_url":"git@github.com:hu-weihua/mig-e2e.git","clone_url":"https://github.com/hu-weihua/mig-e2e.git","svn_url":"https://github.com/hu-weihua/mig-e2e","homepage":null,"size":211,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":228530382,"node_id":"MDEwOlJlcG9zaXRvcnkyMjg1MzAzODI=","name":"mig-operator","full_name":"hu-weihua/mig-operator","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/mig-operator","description":"OpenShift Migration Operator","fork":true,"url":"https://api.github.com/repos/hu-weihua/mig-operator","forks_url":"https://api.github.com/repos/hu-weihua/mig-operator/forks","keys_url":"https://api.github.com/repos/hu-weihua/mig-operator/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/mig-operator/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/mig-operator/teams","hooks_url":"https://api.github.com/repos/hu-weihua/mig-operator/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/mig-operator/events","assignees_url":"https://api.github.com/repos/hu-weihua/mig-operator/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/mig-operator/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/mig-operator/tags","blobs_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/mig-operator/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/mig-operator/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/mig-operator/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/mig-operator/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/mig-operator/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/mig-operator/subscription","commits_url":"https://api.github.com/repos/hu-weihua/mig-operator/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/mig-operator/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/mig-operator/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/mig-operator/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/mig-operator/merges","archive_url":"https://api.github.com/repos/hu-weihua/mig-operator/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/mig-operator/downloads","issues_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/mig-operator/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/mig-operator/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/mig-operator/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/mig-operator/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/mig-operator/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/mig-operator/deployments","created_at":"2019-12-17T04:09:11Z","updated_at":"2020-02-25T14:54:53Z","pushed_at":"2019-12-16T22:52:14Z","git_url":"git://github.com/hu-weihua/mig-operator.git","ssh_url":"git@github.com:hu-weihua/mig-operator.git","clone_url":"https://github.com/hu-weihua/mig-operator.git","svn_url":"https://github.com/hu-weihua/mig-operator","homepage":"","size":15667,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":235289269,"node_id":"MDEwOlJlcG9zaXRvcnkyMzUyODkyNjk=","name":"cam_test","full_name":"hu-weihua/cam_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/cam_test","description":"just for cam test","fork":false,"url":"https://api.github.com/repos/hu-weihua/cam_test","forks_url":"https://api.github.com/repos/hu-weihua/cam_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/cam_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/cam_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/cam_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/cam_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/cam_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/cam_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/cam_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/cam_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/cam_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/cam_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/cam_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/cam_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/cam_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/cam_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/cam_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/cam_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/cam_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/cam_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/cam_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/cam_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/cam_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/cam_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/cam_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/cam_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/cam_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/cam_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/cam_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/cam_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/cam_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/cam_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/cam_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/cam_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/cam_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/cam_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/cam_test/deployments","created_at":"2020-01-21T08:21:49Z","updated_at":"2020-01-21T11:46:02Z","pushed_at":"2020-01-21T11:46:00Z","git_url":"git://github.com/hu-weihua/cam_test.git","ssh_url":"git@github.com:hu-weihua/cam_test.git","clone_url":"https://github.com/hu-weihua/cam_test.git","svn_url":"https://github.com/hu-weihua/cam_test","homepage":null,"size":13,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":231495393,"node_id":"MDEwOlJlcG9zaXRvcnkyMzE0OTUzOTM=","name":"Ansible_projects","full_name":"hu-weihua/Ansible_projects","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/Ansible_projects","description":"To save some ansible projects","fork":false,"url":"https://api.github.com/repos/hu-weihua/Ansible_projects","forks_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/forks","keys_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/teams","hooks_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/events","assignees_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/tags","blobs_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/subscription","commits_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/merges","archive_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/downloads","issues_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/deployments","created_at":"2020-01-03T02:20:16Z","updated_at":"2020-01-03T09:09:29Z","pushed_at":"2020-01-03T09:09:27Z","git_url":"git://github.com/hu-weihua/Ansible_projects.git","ssh_url":"git@github.com:hu-weihua/Ansible_projects.git","clone_url":"https://github.com/hu-weihua/Ansible_projects.git","svn_url":"https://github.com/hu-weihua/Ansible_projects","homepage":null,"size":19,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":88809435,"node_id":"MDEwOlJlcG9zaXRvcnk4ODgwOTQzNQ==","name":"xcat-process","full_name":"hu-weihua/xcat-process","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-process","description":"The process, guidence, rule, tips need to follow by xCAT developers and FVTs","fork":false,"url":"https://api.github.com/repos/hu-weihua/xcat-process","forks_url":"https://api.github.com/repos/hu-weihua/xcat-process/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-process/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-process/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-process/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-process/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-process/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-process/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-process/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-process/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-process/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-process/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-process/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-process/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-process/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-process/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-process/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-process/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-process/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-process/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-process/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-process/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-process/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-process/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-process/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-process/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-process/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-process/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-process/deployments","created_at":"2017-04-20T01:58:16Z","updated_at":"2019-06-25T18:55:32Z","pushed_at":"2017-04-19T09:45:48Z","git_url":"git://github.com/hu-weihua/xcat-process.git","ssh_url":"git@github.com:hu-weihua/xcat-process.git","clone_url":"https://github.com/hu-weihua/xcat-process.git","svn_url":"https://github.com/hu-weihua/xcat-process","homepage":null,"size":337,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":2,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":2,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":40430481,"node_id":"MDEwOlJlcG9zaXRvcnk0MDQzMDQ4MQ==","name":"xcat-core","full_name":"hu-weihua/xcat-core","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-core","description":"Code repo for xCAT core packages","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-core","forks_url":"https://api.github.com/repos/hu-weihua/xcat-core/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-core/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-core/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-core/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-core/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-core/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-core/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-core/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-core/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-core/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-core/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-core/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-core/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-core/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-core/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-core/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-core/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-core/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-core/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-core/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-core/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-core/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-core/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-core/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-core/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-core/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-core/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-core/deployments","created_at":"2015-08-09T08:43:37Z","updated_at":"2019-05-23T09:48:55Z","pushed_at":"2019-05-23T09:48:47Z","git_url":"git://github.com/hu-weihua/xcat-core.git","ssh_url":"git@github.com:hu-weihua/xcat-core.git","clone_url":"https://github.com/hu-weihua/xcat-core.git","svn_url":"https://github.com/hu-weihua/xcat-core","homepage":null,"size":204915,"stargazers_count":0,"watchers_count":0,"language":"Perl","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":184030584,"node_id":"MDEwOlJlcG9zaXRvcnkxODQwMzA1ODQ=","name":"xcat-docker","full_name":"hu-weihua/xcat-docker","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-docker","description":"Repository to build Docker image for xCAT","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-docker","forks_url":"https://api.github.com/repos/hu-weihua/xcat-docker/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-docker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-docker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-docker/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-docker/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-docker/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-docker/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-docker/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-docker/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-docker/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-docker/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-docker/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-docker/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-docker/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-docker/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-docker/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-docker/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-docker/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-docker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-docker/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-docker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-docker/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-docker/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-docker/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-docker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-docker/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-docker/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-docker/deployments","created_at":"2019-04-29T08:25:05Z","updated_at":"2019-05-17T10:33:30Z","pushed_at":"2019-05-23T10:38:53Z","git_url":"git://github.com/hu-weihua/xcat-docker.git","ssh_url":"git@github.com:hu-weihua/xcat-docker.git","clone_url":"https://github.com/hu-weihua/xcat-docker.git","svn_url":"https://github.com/hu-weihua/xcat-docker","homepage":"","size":22,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":178361168,"node_id":"MDEwOlJlcG9zaXRvcnkxNzgzNjExNjg=","name":"xcat2.github.io","full_name":"hu-weihua/xcat2.github.io","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat2.github.io","description":"The repo manages the xcat2 home page: https://xcat2.github.io/","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat2.github.io","forks_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/deployments","created_at":"2019-03-29T08:09:00Z","updated_at":"2019-03-29T08:09:02Z","pushed_at":"2019-03-29T08:25:09Z","git_url":"git://github.com/hu-weihua/xcat2.github.io.git","ssh_url":"git@github.com:hu-weihua/xcat2.github.io.git","clone_url":"https://github.com/hu-weihua/xcat2.github.io.git","svn_url":"https://github.com/hu-weihua/xcat2.github.io","homepage":"","size":35142,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":91755483,"node_id":"MDEwOlJlcG9zaXRvcnk5MTc1NTQ4Mw==","name":"xcat.org","full_name":"hu-weihua/xcat.org","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat.org","description":"Repository for managing the xcat.org website ","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat.org","forks_url":"https://api.github.com/repos/hu-weihua/xcat.org/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat.org/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat.org/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat.org/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat.org/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat.org/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat.org/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat.org/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat.org/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat.org/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat.org/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat.org/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat.org/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat.org/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat.org/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat.org/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat.org/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat.org/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat.org/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat.org/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat.org/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat.org/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat.org/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat.org/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat.org/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat.org/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat.org/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat.org/deployments","created_at":"2017-05-19T02:07:52Z","updated_at":"2018-09-27T06:11:32Z","pushed_at":"2018-09-27T06:11:31Z","git_url":"git://github.com/hu-weihua/xcat.org.git","ssh_url":"git@github.com:hu-weihua/xcat.org.git","clone_url":"https://github.com/hu-weihua/xcat.org.git","svn_url":"https://github.com/hu-weihua/xcat.org","homepage":null,"size":159,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":144540798,"node_id":"MDEwOlJlcG9zaXRvcnkxNDQ1NDA3OTg=","name":"travis_ci_test","full_name":"hu-weihua/travis_ci_test","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/travis_ci_test","description":"This repository is used to test travis ci","fork":false,"url":"https://api.github.com/repos/hu-weihua/travis_ci_test","forks_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/deployments","created_at":"2018-08-13T06:54:07Z","updated_at":"2018-08-17T08:53:39Z","pushed_at":"2018-08-17T08:53:38Z","git_url":"git://github.com/hu-weihua/travis_ci_test.git","ssh_url":"git@github.com:hu-weihua/travis_ci_test.git","clone_url":"https://github.com/hu-weihua/travis_ci_test.git","svn_url":"https://github.com/hu-weihua/travis_ci_test","homepage":null,"size":22,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"has_discussions":false,"forks_count":1,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":1,"open_issues":1,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":131094324,"node_id":"MDEwOlJlcG9zaXRvcnkxMzEwOTQzMjQ=","name":"goconserver","full_name":"hu-weihua/goconserver","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/goconserver","description":"Microservice based console server to help log and redirect the terminal content for multiple session hosts.","fork":true,"url":"https://api.github.com/repos/hu-weihua/goconserver","forks_url":"https://api.github.com/repos/hu-weihua/goconserver/forks","keys_url":"https://api.github.com/repos/hu-weihua/goconserver/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/goconserver/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/goconserver/teams","hooks_url":"https://api.github.com/repos/hu-weihua/goconserver/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/goconserver/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/goconserver/events","assignees_url":"https://api.github.com/repos/hu-weihua/goconserver/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/goconserver/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/goconserver/tags","blobs_url":"https://api.github.com/repos/hu-weihua/goconserver/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/goconserver/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/goconserver/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/goconserver/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/goconserver/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/goconserver/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/goconserver/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/goconserver/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/goconserver/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/goconserver/subscription","commits_url":"https://api.github.com/repos/hu-weihua/goconserver/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/goconserver/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/goconserver/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/goconserver/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/goconserver/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/goconserver/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/goconserver/merges","archive_url":"https://api.github.com/repos/hu-weihua/goconserver/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/goconserver/downloads","issues_url":"https://api.github.com/repos/hu-weihua/goconserver/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/goconserver/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/goconserver/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/goconserver/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/goconserver/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/goconserver/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/goconserver/deployments","created_at":"2018-04-26T03:21:26Z","updated_at":"2018-04-26T03:21:28Z","pushed_at":"2018-04-17T04:11:25Z","git_url":"git://github.com/hu-weihua/goconserver.git","ssh_url":"git@github.com:hu-weihua/goconserver.git","clone_url":"https://github.com/hu-weihua/goconserver.git","svn_url":"https://github.com/hu-weihua/goconserver","homepage":"","size":6913,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":131093726,"node_id":"MDEwOlJlcG9zaXRvcnkxMzEwOTM3MjY=","name":"xcat-inventory","full_name":"hu-weihua/xcat-inventory","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-inventory","description":"an inventory tool for xcat cluster","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-inventory","forks_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/deployments","created_at":"2018-04-26T03:15:17Z","updated_at":"2018-04-26T03:15:19Z","pushed_at":"2018-04-24T20:30:44Z","git_url":"git://github.com/hu-weihua/xcat-inventory.git","ssh_url":"git@github.com:hu-weihua/xcat-inventory.git","clone_url":"https://github.com/hu-weihua/xcat-inventory.git","svn_url":"https://github.com/hu-weihua/xcat-inventory","homepage":null,"size":357,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":40395956,"node_id":"MDEwOlJlcG9zaXRvcnk0MDM5NTk1Ng==","name":"xcat-doc","full_name":"hu-weihua/xcat-doc","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-doc","description":"xCAT Documentation","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-doc","forks_url":"https://api.github.com/repos/hu-weihua/xcat-doc/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-doc/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-doc/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-doc/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-doc/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-doc/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-doc/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-doc/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-doc/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-doc/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-doc/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-doc/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-doc/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-doc/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-doc/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-doc/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-doc/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-doc/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-doc/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-doc/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-doc/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-doc/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-doc/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-doc/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-doc/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-doc/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-doc/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-doc/deployments","created_at":"2015-08-08T07:59:40Z","updated_at":"2015-07-23T02:45:40Z","pushed_at":"2015-08-10T07:06:39Z","git_url":"git://github.com/hu-weihua/xcat-doc.git","ssh_url":"git@github.com:hu-weihua/xcat-doc.git","clone_url":"https://github.com/hu-weihua/xcat-doc.git","svn_url":"https://gith gunicorn-web stdout | ub.com/hu-weihua/xcat-doc","homepage":null,"size":4276,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}}] gunicorn-web stdout | 2023-08-08 05:56:34,833 [200] [DEBUG] [app] Ending request: urn:request:cde571e7-40a8-4502-81ba-8ba658284fd6 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/sources) {'endpoint': 'api.buildtriggersources', 'request_id': 'urn:request:cde571e7-40a8-4502-81ba-8ba658284fd6', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/sources', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/sources', 'parameters': {}, 'json_body': {'namespace': 'hu-weihua'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:34,834 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:34,834 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:34 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/sources HTTP/1.0" 200 6085 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:34 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/sources HTTP/2.0" 200 6085 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.461 565 0.461) notificationworker stdout | 2023-08-08 05:56:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:56:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:45.371682+00:00 (in 9.999510 seconds) notificationworker stdout | 2023-08-08 05:56:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:45 UTC)" (scheduled at 2023-08-08 05:56:35.371682+00:00) notificationworker stdout | 2023-08-08 05:56:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:56:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 35, 372526), True, datetime.datetime(2023, 8, 8, 5, 56, 35, 372526), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:56:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:56:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:56:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:56:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:56:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:54.393627+00:00 (in 17.813283 seconds) namespacegcworker stdout | 2023-08-08 05:56:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:36 UTC)" (scheduled at 2023-08-08 05:56:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:56:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:56:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:37,179 [197] [DEBUG] [app] Starting request: urn:request:518b3f0c-9fdf-497d-9d7a-46019881fabd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:56:37,179 [200] [DEBUG] [app] Starting request: urn:request:66b569ba-70e5-4ce8-aabd-3e46893cbc51 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:56:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:56:37,185 [208] [DEBUG] [app] Starting request: urn:request:e0bcf754-1a39-48de-9584-4020b5d154cf (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:56:37,186 [208] [DEBUG] [app] Ending request: urn:request:e0bcf754-1a39-48de-9584-4020b5d154cf (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e0bcf754-1a39-48de-9584-4020b5d154cf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:56:37,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:56:37,186 [203] [DEBUG] [app] Starting request: urn:request:5432135a-d325-42b6-a962-cb002f11b87a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:56:37,187 [203] [DEBUG] [app] Ending request: urn:request:5432135a-d325-42b6-a962-cb002f11b87a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5432135a-d325-42b6-a962-cb002f11b87a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:56:37,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:56:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:37,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:37,191 [200] [DEBUG] [app] Starting request: urn:request:1c5ae56c-0426-4631-8a64-c994fbb305d0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:37,191 [200] [DEBUG] [app] Ending request: urn:request:1c5ae56c-0426-4631-8a64-c994fbb305d0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1c5ae56c-0426-4631-8a64-c994fbb305d0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:56:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:56:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:37,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:56:37,193 [198] [DEBUG] [app] Starting request: urn:request:5fc98062-3f2f-4da7-a749-354ff97ad629 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:37,194 [198] [DEBUG] [app] Ending request: urn:request:5fc98062-3f2f-4da7-a749-354ff97ad629 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5fc98062-3f2f-4da7-a749-354ff97ad629', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:56:37,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:56:37,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:37,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:56:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:56:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:56:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:07.197412+00:00 (in 29.999296 seconds) buildlogsarchiver stdout | 2023-08-08 05:56:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:07 UTC)" (scheduled at 2023-08-08 05:56:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:56:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 56, 37, 198640), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,200 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:56:37,203 [197] [DEBUG] [app] Ending request: urn:request:518b3f0c-9fdf-497d-9d7a-46019881fabd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:518b3f0c-9fdf-497d-9d7a-46019881fabd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:56:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:56:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:56:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) buildlogsarchiver stdout | 2023-08-08 05:56:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:56:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:56:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:37,205 [200] [DEBUG] [app] Ending request: urn:request:66b569ba-70e5-4ce8-aabd-3e46893cbc51 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:66b569ba-70e5-4ce8-aabd-3e46893cbc51', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:56:37,206 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:56:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) gunicorn-web stdout | 2023-08-08 05:56:37,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:56:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:56:37,673 [199] [DEBUG] [app] Starting request: urn:request:5ba21aac-51ef-485d-9810-0b20d225876e (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/subdir) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:37,673 [197] [DEBUG] [app] Starting request: urn:request:f0bd2f7b-c1ec-4e46-8977-af6f646731da (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:37,673 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,673 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,673 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,673 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,674 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,674 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,682 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:37,682 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:37,682 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:37,682 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:37,682 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,682 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,683 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:37,683 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:37,683 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,683 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,683 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:37,683 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:37,683 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:37,683 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:37,683 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:37,683 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:37,685 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:37,686 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:37,694 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:37,694 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:37,695 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,696 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,698 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,699 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,701 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,702 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,704 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,705 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:37,706 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:37,706 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:56:37,707 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:56:37,832 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:37,833 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:37 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4968', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '32', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F2A:2B81:F43FE0:1F0C358:64D1D915'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAVS7A3OBJ4W2JESSPLE2HNEC","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:56:37,842 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:37,843 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:37 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4967', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '33', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F30:593E:ED5D97:1E316D3:64D1D915'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAVS7A3OBJ4W2JESSPLE2HNEC","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:56:37,845 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:56:37,907 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:37,908 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:37 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"4cfa10a4c34b3fc4519e3ef97204ef558181d88187b3aea2a28f8cff9e905938"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4966', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '34', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F2A:2B81:F44000:1F0C38B:64D1D915'} [{"name":"branch1","commit":{"sha":"1d34effde035584623f29666b8f41b42e020ad4a","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/1d34effde035584623f29666b8f41b42e020ad4a"},"protected":false},{"name":"branch2","commit":{"sha":"c6ef6983d9344c211af3fc7312bddccbe7c0dd7c","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/c6ef6983d9344c211af3fc7312bddccbe7c0dd7c"},"protected":false},{"name":"hu-weihua-patch-1","commit":{"sha":"7f19019c0b0bf7eb48b55479771b213e8081c884","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/7f19019c0b0bf7eb48b55479771b213e8081c884"},"protected":false},{"name":"hu-weihua-patch-2","commit":{"sha":"65e01fcd85dfe53a2d52b5f7e188fca3f30daf78","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/65e01fcd85dfe53a2d52b5f7e188fca3f30daf78"},"protected":false},{"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d"},"protected":false}] gunicorn-web stdout | 2023-08-08 05:56:37,911 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:56:37,974 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:37,975 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:37 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4965', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '35', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F34:2B03:EC39AE:1E09AC2:64D1D915'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAVS7A3OBJ4W2JESSPLE2HNEC","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:56:38,033 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:38,033 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:38 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4964', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '36', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F36:6ED2:E5A9F9:1D3CB08:64D1D915'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAQZM6SUWWFKOJ5LMNDE2HNEE","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:56:38,058 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:38,059 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:38 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"4cfa10a4c34b3fc4519e3ef97204ef558181d88187b3aea2a28f8cff9e905938"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4963', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '37', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F34:2B03:EC39D1:1E09B10:64D1D915'} [{"name":"branch1","commit":{"sha":"1d34effde035584623f29666b8f41b42e020ad4a","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/1d34effde035584623f29666b8f41b42e020ad4a"},"protected":false},{"name":"branch2","commit":{"sha":"c6ef6983d9344c211af3fc7312bddccbe7c0dd7c","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/c6ef6983d9344c211af3fc7312bddccbe7c0dd7c"},"protected":false},{"name":"hu-weihua-patch-1","commit":{"sha":"7f19019c0b0bf7eb48b55479771b213e8081c884","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/7f19019c0b0bf7eb48b55479771b213e8081c884"},"protected":false},{"name":"hu-weihua-patch-2","commit":{"sha":"65e01fcd85dfe53a2d52b5f7e188fca3f30daf78","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/65e01fcd85dfe53a2d52b5f7e188fca3f30daf78"},"protected":false},{"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d"},"protected":false}] gunicorn-web stdout | 2023-08-08 05:56:38,095 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/tags HTTP/1.1" 200 2 gunicorn-web stdout | 2023-08-08 05:56:38,095 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/tags {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:38 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '2', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"4c75d91aaf88ebe4a7429173cbe96e13732080c64455c9da24aeaca752887ab8"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4962', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '38', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': '9F36:6ED2:E5AA1A:1D3CB4E:64D1D916'} [] gunicorn-web stdout | 2023-08-08 05:56:38,098 [197] [DEBUG] [app] Ending request: urn:request:f0bd2f7b-c1ec-4e46-8977-af6f646731da (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs) {'endpoint': 'api.buildtriggerfieldvalues', 'request_id': 'urn:request:f0bd2f7b-c1ec-4e46-8977-af6f646731da', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs', 'parameters': {}, 'json_body': {'build_source': 'hu-weihua/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:38,098 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:38,099 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:38 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs HTTP/1.0" 200 225 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:38 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs HTTP/2.0" 200 225 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.428 497 0.427) gunicorn-web stdout | 2023-08-08 05:56:38,173 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches/main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:38,174 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches/main {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:38 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"d8accd6c537e1434f53369c16792470386400303ebf09d6aff58ab4a24589a86"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4961', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '39', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F30:593E:ED5DF7:1E317B1:64D1D915'} {"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","node_id":"C_kwDOG6AIe9oAKDg4MGZiNzAwZjhiNTEwM2U3YWRjMjBmMDgwMzg0MzRlOWY3MTJjNWQ","commit":{"author":{"name":"Hu Weihua","email":"hu_weihua@163.com","date":"2023-08-08T02:50:20Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2023-08-08T02:50:20Z"},"message":"Update update_commit.sh","tree":{"sha":"c5abbb94e6edfb389f732ea578c71d6db692406d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/c5abbb94e6edfb389f732ea578c71d6db692406d"},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsBcBAABCAAQBQJk0a1sCRBK7hj4Ov3rIwAA5MMIABciqM6Y7qngb0Joeh7FSIAp\nnv0pckL5M+0g6IBcVfxG4RC6Hfti5Gs1lYKBppBJMF5OUVdITpe2GOii6WcGieL9\nbo/Acx0X4liVp4QzoQQTNYRmCZFODrlb/FVuVcdHRtBO/hXNAsqmPOHGC1Guf5lG\nHWpLPZF9e8lnbI1/MIiu4O5+OK+qRw26pEbRHjukBa921OQ/27s+MATflSKJ+bBJ\nBshN6stBgyAA4kDsPwl7iIHn1xSGhADLw3q8BnDRIT5jRHzDoqHdck9VpVatNxxL\nK+9HZiaHhyQ3+CsvT5/NNMy5cWoqJvSpjNobgmw4bYg70AB2xUHy85jPOl7Rizk=\n=DSm6\n-----END PGP SIGNATURE-----\n","payload":"tree c5abbb94e6edfb389f732ea578c71d6db692406d\nparent f0030ac4a54f1c210daf84729cfd233407d43d14\nauthor Hu Weihua 1691463020 +0800\ncommitter GitHub 1691463020 +0800\n\nUpdate update_commit.sh"}},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","html_url":"https://github.com/hu-weihua/quay_build_test/commit/880fb700f8b5103e7adc20f08038434e9f712c5d","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d/comments","author":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","site_admin":false},"parents":[{"sha":"f0030ac4a54f1c210daf84729cfd233407d43d14","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/f0030ac4a54f1c210daf84729cfd233407d43d14","html_url":"https://github.com/hu-weihua/quay_build_test/commit/f0030ac4a54f1c210daf84729cfd233407d43d14"}]},"_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/branches/main","html":"https://github.com/hu-weihua/quay_build_test/tree/main"},"protected":false,"protection":{"enabled":false,"required_status_checks":{"enforcement_level":"off","contexts":[],"checks":[]}},"protection_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches/main/protection"} gunicorn-web stdout | 2023-08-08 05:56:38,274 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d?recursive=1 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:38,275 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d?recursive=1 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:38 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=86400, s-maxage=86400', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2ce4599de9d2091c586c95192895546bdad15e355c028ee72a93a2cdc120927a"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4960', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '40', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '9F30:593E:ED5E25:1E31804:64D1D916'} {"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d","tree":[{"path":"Dockerfile","mode":"100644","type":"blob","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6"},{"path":"README.md","mode":"100644","type":"blob","sha":"929a3f9add2efe6f5d73c5226dc603686c63a202","size":57,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/929a3f9add2efe6f5d73c5226dc603686c63a202"},{"path":"aaa","mode":"100644","type":"blob","sha":"73cf2b7c923ab8e99ff19403d3b7c4191b2460a3","size":20,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/73cf2b7c923ab8e99ff19403d3b7c4191b2460a3"},{"path":"newfile5540167606144473","mode":"100644","type":"blob","sha":"aa5d92554efab5c3075f098e51bd8dc9642d6075","size":22,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/aa5d92554efab5c3075f098e51bd8dc9642d6075"},{"path":"newfile7189991658911539","mode":"100644","type":"blob","sha":"894bdc5476f8f12f6e90c706cc91aa84cf327231","size":22,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/894bdc5476f8f12f6e90c706cc91aa84cf327231"},{"path":"update_commit.sh","mode":"100644","type":"blob","sha":"8ac65cd6da3c09ee9b225fddc5eea5023f65c6aa","size":69,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/8ac65cd6da3c09ee9b225fddc5eea5023f65c6aa"}],"truncated":false} gunicorn-web stdout | 2023-08-08 05:56:38,277 [199] [DEBUG] [app] Ending request: urn:request:5ba21aac-51ef-485d-9810-0b20d225876e (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/subdir) {'endpoint': 'api.buildtriggersubdirs', 'request_id': 'urn:request:5ba21aac-51ef-485d-9810-0b20d225876e', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/subdir', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/subdir', 'parameters': {}, 'json_body': {'build_source': 'hu-weihua/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:38,278 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:38 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/subdir HTTP/2.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.607 402 0.607) gunicorn-web stdout | 2023-08-08 05:56:38,278 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:38 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/subdir HTTP/1.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gcworker stdout | 2023-08-08 05:56:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:56:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:49.417660+00:00 (in 9.999438 seconds) gcworker stdout | 2023-08-08 05:56:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:49 UTC)" (scheduled at 2023-08-08 05:56:39.417660+00:00) gcworker stdout | 2023-08-08 05:56:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264599418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:56:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:56:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:56:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:49 UTC)" executed successfully builder stdout | 2023-08-08 05:56:40,664 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:56:40,673 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:56:40,673 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:56:40,678 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:56:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:56:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:52.293757+00:00 (in 9.999504 seconds) chunkcleanupworker stdout | 2023-08-08 05:56:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:52 UTC)" (scheduled at 2023-08-08 05:56:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:56:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:56:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 42, 294740), True, datetime.datetime(2023, 8, 8, 5, 56, 42, 294740), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:56:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:56:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:56:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:52 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:44,388 [200] [DEBUG] [app] Starting request: urn:request:ef6c6c4a-3ed9-4f21-9ff8-601b99ce5baa (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:44,388 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:44,388 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:44,389 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:44,396 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:44,396 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:44,396 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:44,397 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:44,397 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:44,397 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:44,397 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:44,397 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:44,399 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:44,406 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:44,407 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:44,410 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:44,413 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:44,416 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:44,417 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:56:44,419 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:56:44,419 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:56:44,419 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:56:44,419 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:56:44,421 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:56:44,566 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:44,567 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:44 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4959', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '41', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'CC66:0C81:E9A6AD:1DB8BAC:64D1D91C'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAR47NBHWKRLIVINXYDE2HNEQ","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:56:44,650 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/contents/Dockerfile HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:44,651 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:44 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"35a89d24e0791670dc4732e5402e3a749e89c5e6"', 'last-modified': 'Tue, 08 Aug 2023 02:50:20 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4958', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '42', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'CC66:0C81:E9A6D5:1DB8C00:64D1D91C'} {"name":"Dockerfile","path":"Dockerfile","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","html_url":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile","git_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","download_url":"https://raw.githubusercontent.com/hu-weihua/quay_build_test/main/Dockerfile?token=ABYHPAVCNE5W6S44GEG4UJTE2HMVQ","type":"file","content":"RlJPTSByZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbS91Ymk3L3ViaQoKVVNF\nUiByb290CkxBQkVMIG1haW50YWluZXI9IndodSIKCiMgVXBkYXRlIGltYWdl\nClJVTiB5dW0gdXBkYXRlIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24t\nbWFuYWdlciAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KUlVOIHl1bSBp\nbnN0YWxsIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24tbWFuYWdlciBo\ndHRwZCAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KCiMgQWRkIGRlZmF1\nbHQgV2ViIHBhZ2UgYW5kIGV4cG9zZSBwb3J0ClJVTiBlY2hvICJUaGUgV2Vi\nIFNlcnZlciBpcyBSdW5uaW5nIiA+IC92YXIvd3d3L2h0bWwvaW5kZXguaHRt\nbApSVU4gZWNobyAicHVzaCB0ZXN0IiA+IC93aHUudHh0ClJVTiBlY2hvICJw\ndXNoIHRlc3QxIiA+IC93aHUudHh0CkVYUE9TRSA4MAoKIyBTdGFydCB0aGUg\nc2VydmljZQpDTUQgWyItRCIsICJGT1JFR1JPVU5EIl0KRU5UUllQT0lOVCBb\nIi91c3Ivc2Jpbi9odHRwZCJdCg==\n","encoding":"base64","_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","git":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","html":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile"}} gunicorn-web stdout | 2023-08-08 05:56:44,653 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 05:56:44,656 [200] [DEBUG] [app] Ending request: urn:request:ef6c6c4a-3ed9-4f21-9ff8-601b99ce5baa (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:ef6c6c4a-3ed9-4f21-9ff8-601b99ce5baa', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': None}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:44,656 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:44 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze HTTP/2.0" 200 115 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.271 1114 0.270) gunicorn-web stdout | 2023-08-08 05:56:44,657 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:44 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze HTTP/1.0" 200 115 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 05:56:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:56:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:55.371682+00:00 (in 9.999515 seconds) notificationworker stdout | 2023-08-08 05:56:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:55 UTC)" (scheduled at 2023-08-08 05:56:45.371682+00:00) notificationworker stdout | 2023-08-08 05:56:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:56:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 45, 372529), True, datetime.datetime(2023, 8, 8, 5, 56, 45, 372529), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:56:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:56:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:56:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:56:47,285 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:56:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:56:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:25.514341+00:00 (in 38.115214 seconds) exportactionlogsworker stdout | 2023-08-08 05:56:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:47 UTC)" (scheduled at 2023-08-08 05:56:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:56:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:56:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 47, 399703), True, datetime.datetime(2023, 8, 8, 5, 56, 47, 399703), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:56:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:56:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:56:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:56:48,547 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 05:56:49,299 [199] [DEBUG] [app] Starting request: urn:request:67cb1a99-afd6-4d7c-bee8-9c8a6d4a72ed (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:49,299 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:49,299 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:49,300 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:49,307 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:49,307 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:49,307 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:49,307 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:49,307 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:49,307 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:49,307 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:49,307 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:49,310 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:49,316 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:49,317 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:49,320 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:49,323 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:49,325 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:49,326 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 05:56:49,328 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:56:49,328 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 05:56:49,328 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 05:56:49,328 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 05:56:49,330 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gcworker stdout | 2023-08-08 05:56:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:56:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:59.417660+00:00 (in 9.999301 seconds) gcworker stdout | 2023-08-08 05:56:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:59 UTC)" (scheduled at 2023-08-08 05:56:49.417660+00:00) gcworker stdout | 2023-08-08 05:56:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264609418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:56:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:56:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:56:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:56:59 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:49,465 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:49,465 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:49 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4957', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '43', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'BA3D:900F:E0FC44:1CA5B74:64D1D921'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAWWQ6RBBUNNZ46MPWLE2HNE2","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:56:49,541 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/contents/Dockerfile HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:49,542 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:49 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"35a89d24e0791670dc4732e5402e3a749e89c5e6"', 'last-modified': 'Tue, 08 Aug 2023 02:50:20 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4956', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '44', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'BA3D:900F:E0FC6D:1CA5BB5:64D1D921'} {"name":"Dockerfile","path":"Dockerfile","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","html_url":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile","git_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","download_url":"https://raw.githubusercontent.com/hu-weihua/quay_build_test/main/Dockerfile?token=ABYHPAUBBG47IOC5MJC6W6LE2HMV2","type":"file","content":"RlJPTSByZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbS91Ymk3L3ViaQoKVVNF\nUiByb290CkxBQkVMIG1haW50YWluZXI9IndodSIKCiMgVXBkYXRlIGltYWdl\nClJVTiB5dW0gdXBkYXRlIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24t\nbWFuYWdlciAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KUlVOIHl1bSBp\nbnN0YWxsIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24tbWFuYWdlciBo\ndHRwZCAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KCiMgQWRkIGRlZmF1\nbHQgV2ViIHBhZ2UgYW5kIGV4cG9zZSBwb3J0ClJVTiBlY2hvICJUaGUgV2Vi\nIFNlcnZlciBpcyBSdW5uaW5nIiA+IC92YXIvd3d3L2h0bWwvaW5kZXguaHRt\nbApSVU4gZWNobyAicHVzaCB0ZXN0IiA+IC93aHUudHh0ClJVTiBlY2hvICJw\ndXNoIHRlc3QxIiA+IC93aHUudHh0CkVYUE9TRSA4MAoKIyBTdGFydCB0aGUg\nc2VydmljZQpDTUQgWyItRCIsICJGT1JFR1JPVU5EIl0KRU5UUllQT0lOVCBb\nIi91c3Ivc2Jpbi9odHRwZCJdCg==\n","encoding":"base64","_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","git":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","html":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile"}} gunicorn-web stdout | 2023-08-08 05:56:49,545 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 05:56:49,550 [199] [DEBUG] [app] Ending request: urn:request:67cb1a99-afd6-4d7c-bee8-9c8a6d4a72ed (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:67cb1a99-afd6-4d7c-bee8-9c8a6d4a72ed', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:49,550 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:49 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze HTTP/2.0" 200 115 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.254 550 0.253) gunicorn-web stdout | 2023-08-08 05:56:49,551 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:49 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/analyze HTTP/1.0" 200 115 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityworker stdout | 2023-08-08 05:56:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:56:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:56:56.890741+00:00 (in 7.202846 seconds) securityworker stdout | 2023-08-08 05:56:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:19 UTC)" (scheduled at 2023-08-08 05:56:49.687363+00:00) securityworker stdout | 2023-08-08 05:56:49,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:56:49,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:56:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:56:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:56:49,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:56:49,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:56:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:56:49,700 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 05:56:50,049 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:56:50,688 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:56:50,697 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:56:50,697 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:56:50,701 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:52,179 [198] [DEBUG] [app] Starting request: urn:request:fd790858-56a8-40f5-8d18-b91669928073 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:56:52,179 [200] [DEBUG] [app] Starting request: urn:request:3531090d-d339-4692-ad1c-823974c04765 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:56:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:52,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:56:52,184 [203] [DEBUG] [app] Starting request: urn:request:1460ab91-d95c-4c80-8d06-a008fa160f60 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:56:52,185 [210] [DEBUG] [app] Starting request: urn:request:6aecedd7-3018-451c-821c-02c80d462f09 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:56:52,185 [203] [DEBUG] [app] Ending request: urn:request:1460ab91-d95c-4c80-8d06-a008fa160f60 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1460ab91-d95c-4c80-8d06-a008fa160f60', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:56:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:56:52,185 [210] [DEBUG] [app] Ending request: urn:request:6aecedd7-3018-451c-821c-02c80d462f09 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6aecedd7-3018-451c-821c-02c80d462f09', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:56:52,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:56:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:56:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:56:52,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:56:52,190 [199] [DEBUG] [app] Starting request: urn:request:dace4bf1-9c31-4c09-8107-4876725d1b71 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:52,190 [197] [DEBUG] [app] Starting request: urn:request:e5e3251a-b70c-43aa-8f95-7a8804350446 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:56:52,191 [199] [DEBUG] [app] Ending request: urn:request:dace4bf1-9c31-4c09-8107-4876725d1b71 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:dace4bf1-9c31-4c09-8107-4876725d1b71', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:56:52,191 [197] [DEBUG] [app] Ending request: urn:request:e5e3251a-b70c-43aa-8f95-7a8804350446 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e5e3251a-b70c-43aa-8f95-7a8804350446', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:56:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:56:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:56:52,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:56:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:56:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:52,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:56:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:56:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:56:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:56:52,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:56:52,203 [200] [DEBUG] [app] Ending request: urn:request:3531090d-d339-4692-ad1c-823974c04765 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3531090d-d339-4692-ad1c-823974c04765', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:56:52,203 [198] [DEBUG] [app] Ending request: urn:request:fd790858-56a8-40f5-8d18-b91669928073 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fd790858-56a8-40f5-8d18-b91669928073', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:56:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:56:52,203 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:56:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:56:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 05:56:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:56:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:56:52,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:56:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:56:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:56:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:02.293757+00:00 (in 9.999459 seconds) chunkcleanupworker stdout | 2023-08-08 05:56:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:02 UTC)" (scheduled at 2023-08-08 05:56:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:56:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:56:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 52, 294860), True, datetime.datetime(2023, 8, 8, 5, 56, 52, 294860), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:56:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:56:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:56:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:56:52,842 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:56:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:56:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:13.097375+00:00 (in 18.812686 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:56:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:54 UTC)" (scheduled at 2023-08-08 05:56:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:56:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:56:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:56:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:56:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:36.579841+00:00 (in 42.185730 seconds) namespacegcworker stdout | 2023-08-08 05:56:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:54 UTC)" (scheduled at 2023-08-08 05:56:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:56:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:56:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 54, 394571), True, datetime.datetime(2023, 8, 8, 5, 56, 54, 394571), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:56:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:56:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:56:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:57:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:56:54,661 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:56:54,753 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:56:54,787 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:56:54,790 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:56:54,792 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:56:55,042 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:56:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:56:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:02.178530+00:00 (in 6.806349 seconds) notificationworker stdout | 2023-08-08 05:56:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:05 UTC)" (scheduled at 2023-08-08 05:56:55.371682+00:00) notificationworker stdout | 2023-08-08 05:56:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:56:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 56, 55, 372538), True, datetime.datetime(2023, 8, 8, 5, 56, 55, 372538), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:56:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:56:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:56:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:05 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:55,896 [200] [DEBUG] [app] Starting request: urn:request:7bbb3649-5517-48d8-9c18-59c4d26d5917 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/activate) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:56:55,897 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:55,897 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:55,897 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:55,906 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:56:55,906 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:56:55,906 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:55,906 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:56:55,906 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:55,906 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:55,906 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:55,906 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:56:55,909 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:56:55,917 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:56:55,918 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:55,921 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:55,924 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:55,927 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:55,928 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:56:55,928 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['write', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:55,930 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:55,932 [200] [DEBUG] [peewee] ('INSERT INTO "accesstoken" ("friendly_name", "token_name", "token_code", "repository_id", "created", "role_id", "temporary") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "accesstoken"."id"', ['Build Trigger: github', '84QTNLIUTFL6LS45SECGGJDC6DE8MQWV', 'v0$$9r+NxfIjqFegddeD7V1uHl+sK3zNozKbbVmnHmq8h8ZU+bv2AASW+urUZvqYDl1GO3BrZD7zUi8DKIKFBg==', 2, datetime.datetime(2023, 8, 8, 5, 56, 55, 931940), 2, False]) gunicorn-web stdout | 2023-08-08 05:56:55,937 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:56:56,060 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:56:56,061 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:56 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4955', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '45', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'C5EA:6ED2:E5C30C:1D3FE6D:64D1D927'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAXA7ZGUQ7BR5JXM5C3E2HNFI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} teamsyncworker stdout | 2023-08-08 05:56:56,137 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 05:56:56,256 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "POST /repos/hu-weihua/quay_build_test/keys HTTP/1.1" 201 622 gunicorn-web stdout | 2023-08-08 05:56:56,257 [200] [DEBUG] [github.Requester] POST https://api.github.com/repos/hu-weihua/quay_build_test/keys {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python', 'Content-Type': 'application/json'} {"title": "Red Hat Quay Builder", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC63lMfzSR0NmndWK5RZOSYTWBnO8GkaU92QfwuVFlzjWr/Q+SVO75VlMQ7sueSw7AnGUmJPMY3WPAX/GK5wkLorz/GMzBq/LX2MK+k+jJ8/qz/jlqViIwIKScpUSOUb7t6s1ZF+08zhg4AstS2/Hbo8wrfPwgjEI2W01PfTYSXi6KmOWL3Eisy1SlSEtquB6jAbqsSWY1Rq5iHHDJyKBhHZ/ZHVc6pkw4PI3hdQO8GBG0y0iSAWTulAyTHA1LU3ucZEAuYZ99l1rkEI9TOHrxxx1PX/b0y+EjvXknwMHQBHgWOA/y93HYmT1uN6o4TP/tZKfjTMDmykHijwkBgz5+V", "read_only": false} ==> 201 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:56 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '622', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"8c8ac0247bb39746ee1d0aa86d10323b416a9df3e81998f0156eb2cb796a3b2f"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'location': 'https://api.github.com/repos/hu-weihua/quay_build_test/keys/85930234', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4954', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '46', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': 'C5EA:6ED2:E5C339:1D3FEC7:64D1D928'} {"id":85930234,"key":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC63lMfzSR0NmndWK5RZOSYTWBnO8GkaU92QfwuVFlzjWr/Q+SVO75VlMQ7sueSw7AnGUmJPMY3WPAX/GK5wkLorz/GMzBq/LX2MK+k+jJ8/qz/jlqViIwIKScpUSOUb7t6s1ZF+08zhg4AstS2/Hbo8wrfPwgjEI2W01PfTYSXi6KmOWL3Eisy1SlSEtquB6jAbqsSWY1Rq5iHHDJyKBhHZ/ZHVc6pkw4PI3hdQO8GBG0y0iSAWTulAyTHA1LU3ucZEAuYZ99l1rkEI9TOHrxxx1PX/b0y+EjvXknwMHQBHgWOA/y93HYmT1uN6o4TP/tZKfjTMDmykHijwkBgz5+V","url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys/85930234","title":"Red Hat Quay Builder","verified":true,"created_at":"2023-08-08T05:56:56Z","read_only":false,"last_used":null,"added_by":"hu-weihua"} gunicorn-web stdout | 2023-08-08 05:56:56,408 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "POST /repos/hu-weihua/quay_build_test/hooks HTTP/1.1" 201 853 gunicorn-web stdout | 2023-08-08 05:56:56,408 [200] [DEBUG] [github.Requester] POST https://api.github.com/repos/hu-weihua/quay_build_test/hooks {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python', 'Content-Type': 'application/json'} {"name": "web", "config": {"url": "https://$token:84QTNLIUTFL6LS45SECGGJDC6DE8MQWVV3QKZOJ20EXPZC2HMU78JOQPA3P7MCPG@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b", "content_type": "json"}} ==> 201 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:56:56 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '853', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"7935a99811bad4b723ebb0af0c2c3b7901328df040393db5d503c73f645d7de3"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:repo_hook, repo, write:repo_hook', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'location': 'https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817855', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4953', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '47', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': 'C5EA:6ED2:E5C37D:1D3FF47:64D1D928'} {"type":"Repository","id":427817855,"name":"web","active":true,"events":["push"],"config":{"content_type":"json","insecure_ssl":"0","url":"https://$token:84QTNLIUTFL6LS45SECGGJDC6DE8MQWVV3QKZOJ20EXPZC2HMU78JOQPA3P7MCPG@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b"},"updated_at":"2023-08-08T05:56:56Z","created_at":"2023-08-08T05:56:56Z","url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817855","test_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817855/test","ping_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817855/pings","deliveries_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427817855/deliveries","last_response":{"code":null,"status":"unused","message":null}} gunicorn-web stdout | 2023-08-08 05:56:56,411 [200] [DEBUG] [peewee] ('UPDATE "repositorybuildtrigger" SET "uuid" = %s, "service_id" = %s, "repository_id" = %s, "connected_user_id" = %s, "secure_auth_token" = %s, "secure_private_key" = %s, "fully_migrated" = %s, "config" = %s, "write_token_id" = %s, "pull_robot_id" = %s, "enabled" = %s, "disabled_reason_id" = %s, "disabled_datetime" = %s, "successive_failure_count" = %s, "successive_internal_error_count" = %s WHERE ("repositorybuildtrigger"."id" = %s)', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 2, 1, 'v0$$5V29IBxCwbRzoUWHFpD1NiZkzW/p9zx8uAKuMCjrK5Q2K0AUwYSRNHE8IvOu+AlW1LluSDoW9j4944XcROydzrkDEyrb', 'v0$$dhoK9gFOo6hI6EUBCKmWMhBOsPiB1Fl2jMprkZEVUGj960mdGeqUHAmUXKKa9dwJR2i4Jzi5pbPswjKWLD/i5xZslXhApbMmoVraBvgHopMYKC4sQTRjZk++a/BOsPv1upAPEjF3KiCDcK41vYINpfe5AtGkk9kKUEuwOfRNjeNvyRGGWwrtdO//69k7Zg7Aw3neu85fd2SBx5K4jVSNG3kWRZrGWgYwsgt0G6XCEwkIaw2BWUBA9CKBnrkDW1MstIteMjPZNJgLCt8frVvfe7We8DzfZG7Mo7Ca2SMAsrymWp+6ghLQDD1jvJtMcQ/YARWx5cM65aCOD7fadPtfEqgKAy8EnpAl83ReyvYfiAmlWfOCR7BLpnB3VfuMNt7QcWhXNCf1jeuW57E+hHE1VWwCExbHAcY7BcMCKMgQ7NVnu2qwWPVd9DhtLEXkt+U0fjbKemtJL4ehaqZm6AAacOT1AvoJHLOPhwfS+eTUoBZNSSSCpCHrnB9Q6A03NwkygOVKyv/a2VQU7yEMjmH4xWg+kkdeA+PnPapQO+m2EKAjyB4JCsLBYM3dDGCA0QakvmJZX1DaFZhzoCfwk4Lry+eFu5CiCufFablPQZm2/68Yvdbm5Kvxl1zfR5drAFl8YmZXdiZBQ3soOs/ZTy26bCEY6XXh+WaAJi+ZjPRY7pCGKYfT3EzSZqCLhEmWWRb1B1oB9M6iWcznjeJCnJtS5gekqKdzVoxYqWeIYNN/66drZmPngMa1BK4yLKlF7m6JP9fmNvcAbacFk2+w+VRnV0wBp/xdU2gdIkgUcZmPoCvy/bJ26jSDYv0Jz1L4tlfDKXDvWTnFkWUcUMJt1+tgPwOSzhkstkBoiS9r54CP9oqfilu2w3tyTq1/DZCWIQ+p3qBzXHVyGQaJSfqB0aqeddTs1Cxp3gro7H7KdLuDN6lkjLNMRfvucEEp8jlqZw3Ej/o8tOu/GSRRdJo+eaUMb7DH6EkUvQUWHftH6NUdCsF0oSElQOd9wqka1OeERr6hy2w/yRHLPwa1Q1RXZUp/jdYHHAaaTJaev1muHlDH+rRxFCibw8YxcftlddZiokNEfPSBn4o0aRI8YXRnXyR8P0bhR8XhqrtvjCNbZboQ/T46X2XYUEf33uF5ddQcjKVmnhs3smSkou2aXMouoIiaOvD+afzH8OtdF8Brnqmca9HRBJ2Ng7hUvraWNAbWlwpYyJ9+SdK3XsqfWp7X9Z+ZDfahw4ac3OObrtU9m3cmQyiB4dFA4/VS/lBkqd+u5c7hWyno5H09oELceTtfvbNsqpcT6YLR5iMoWA2inKdkKkPQ7ifVrd1cE5lJ5Y8Wu7DKb0oV5ykDDOIqPJj3UOTk86/vAn6vKjLLxX5+kqHNnA2SB53jphnWzlgTpdo0a4jBh5PVl2iz8Bnc/vHkWpbVMkMy/D2agGMvwwun6aOgiaE5gxH6AE5bEDP89uDI7IEt4+kejHJW+cqGG3dt20yr6kHsdproz2uaqc8I9xh0+WWfCm8cTjH96oFN34SRqrcvbvtCtDmrM6lTiJA6ESN1/U2M42IWRSC2DHEEqj83uqCLkO0WX/ik5hl29BpjUyC8dsoruI2tSsvt4hFYRn9VlhfY6ZO67klAh+izWsRWzt7yMh3/SAhJJb+SblInMXaa7PiD7QLW31r3mzmO4/EL1ppv06DTFbQh8afFLh5rI0wtic2Ji2jhgyi1zfMBGOKMBb0630hjP/lLXzdFX8OOj3lJUQeplqDEYHF1JouPwpuez462nkGIIRRhmT2PQiCnIxzzlc+FDZAyrsdxh2H7/W14BcKLah3c6ubKUvwZxBuAOfeSl+MDd0wTMgK3eFcR7UVpAie4qX7FALh/FXvMsgxnk/tgzdA7lXN8k9RoVeVV2C3gA8hPpeBknCi7WkixOMR2iwjzaQ1gtmWdiBL+38yJ+1CaHmGTVrW42fXK5b1tVkdNEmLwDcMamJDsrYBO1StPrv8T6QzW/V4FxFVRn7G2GaIFHb/L7HEJU1XXUeOQBrQ43mKDjVUOtb4o3O6c4p5rHrCoDHFSD2gjjIX72EfafncoB0kMR0l248ag+OiNfZL8NhWsIK3klferSpMcGLiDkQNU6K8MXYm6YVhiln+XqTf7rF+ukRquWnA8lPlWsQSJfkY/fIEbsMrKTYDnRnhktw72vO4KuYEcsG1XxkEQs3SGU64CveKXqFF4oyLlZJKSVqIyGx9uA/STejlrWpzP1BUlLJp9XQo6Z9yb8HflXeYmttnx', False, '{"build_source": "hu-weihua/quay_build_test", "dockerfile_path": "/Dockerfile", "context": "/", "default_tag_from_ref": true, "latest_for_default_branch": true, "tag_templates": [], "credentials": [{"name": "SSH Public Key", "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC63lMfzSR0NmndWK5RZOSYTWBnO8GkaU92QfwuVFlzjWr/Q+SVO75VlMQ7sueSw7AnGUmJPMY3WPAX/GK5wkLorz/GMzBq/LX2MK+k+jJ8/qz/jlqViIwIKScpUSOUb7t6s1ZF+08zhg4AstS2/Hbo8wrfPwgjEI2W01PfTYSXi6KmOWL3Eisy1SlSEtquB6jAbqsSWY1Rq5iHHDJyKBhHZ/ZHVc6pkw4PI3hdQO8GBG0y0iSAWTulAyTHA1LU3ucZEAuYZ99l1rkEI9TOHrxxx1PX/b0y+EjvXknwMHQBHgWOA/y93HYmT1uN6o4TP/tZKfjTMDmykHijwkBgz5+V"}], "deploy_key_id": 85930234, "hook_id": 427817855, "master_branch": "main"}', 3, None, True, None, datetime.datetime(2023, 8, 8, 5, 56, 30, 154069), 0, 0, 3]) gunicorn-web stdout | 2023-08-08 05:56:56,415 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:56,417 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:56:56,419 [200] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [53, 2, 1, 2, datetime.datetime(2023, 8, 8, 5, 56, 56, 417443), '10.130.2.1', '{"repo": "github", "namespace": "build_test", "trigger_id": "3d8c06fe-e700-4fee-91c0-e0e170c77f8b", "service": "github", "pull_robot": null, "config": {"build_source": "hu-weihua/quay_build_test", "dockerfile_path": "/Dockerfile", "context": "/", "default_tag_from_ref": true, "latest_for_default_branch": true, "tag_templates": [], "credentials": [{"name": "SSH Public Key", "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC63lMfzSR0NmndWK5RZOSYTWBnO8GkaU92QfwuVFlzjWr/Q+SVO75VlMQ7sueSw7AnGUmJPMY3WPAX/GK5wkLorz/GMzBq/LX2MK+k+jJ8/qz/jlqViIwIKScpUSOUb7t6s1ZF+08zhg4AstS2/Hbo8wrfPwgjEI2W01PfTYSXi6KmOWL3Eisy1SlSEtquB6jAbqsSWY1Rq5iHHDJyKBhHZ/ZHVc6pkw4PI3hdQO8GBG0y0iSAWTulAyTHA1LU3ucZEAuYZ99l1rkEI9TOHrxxx1PX/b0y+EjvXknwMHQBHgWOA/y93HYmT1uN6o4TP/tZKfjTMDmykHijwkBgz5+V"}], "deploy_key_id": 85930234, "hook_id": 427817855, "master_branch": "main"}}']) gunicorn-web stdout | 2023-08-08 05:56:56,422 [200] [DEBUG] [app] Ending request: urn:request:7bbb3649-5517-48d8-9c18-59c4d26d5917 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/activate) {'endpoint': 'api.buildtriggeractivate', 'request_id': 'urn:request:7bbb3649-5517-48d8-9c18-59c4d26d5917', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/activate', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/activate', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': '/Dockerfile', 'context': '/', 'default_tag_from_ref': True, 'latest_for_default_branch': True, 'tag_templates': [], 'credentials': [{'name': 'SSH Public Key', 'value': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC63lMfzSR0NmndWK5RZOSYTWBnO8GkaU92QfwuVFlzjWr/Q+SVO75VlMQ7sueSw7AnGUmJPMY3WPAX/GK5wkLorz/GMzBq/LX2MK+k+jJ8/qz/jlqViIwIKScpUSOUb7t6s1ZF+08zhg4AstS2/Hbo8wrfPwgjEI2W01PfTYSXi6KmOWL3Eisy1SlSEtquB6jAbqsSWY1Rq5iHHDJyKBhHZ/ZHVc6pkw4PI3hdQO8GBG0y0iSAWTulAyTHA1LU3ucZEAuYZ99l1rkEI9TOHrxxx1PX/b0y+EjvXknwMHQBHgWOA/y93HYmT1uN6o4TP/tZKfjTMDmykHijwkBgz5+V'}], 'deploy_key_id': 85930234, 'hook_id': 427817855, 'master_branch': 'main'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:56:56,423 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:56:56 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/activate HTTP/2.0" 200 965 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.528 1194 0.529) gunicorn-web stdout | 2023-08-08 05:56:56,423 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:56:56 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/activate HTTP/1.0" 200 965 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" queuecleanupworker stdout | 2023-08-08 05:56:56,681 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:56:56,863 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:56:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:56:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:19.687363+00:00 (in 22.796049 seconds) securityworker stdout | 2023-08-08 05:56:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:26 UTC)" (scheduled at 2023-08-08 05:56:56.890741+00:00) securityworker stdout | 2023-08-08 05:56:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:56:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:56:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:56:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:56:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:56:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:56:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:56:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:56:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:56:56,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:56:56,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:56:56,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:56:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:56:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:56:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:56:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:56:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:56:56,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:56:57,193 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:56:57,211 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:56:57,221 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:56:57,232 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:56:57,234 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:56:58,066 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:56:58,239 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:56:59,324 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:56:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:56:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:09.417660+00:00 (in 9.999486 seconds) gcworker stdout | 2023-08-08 05:56:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:09 UTC)" (scheduled at 2023-08-08 05:56:59.417660+00:00) gcworker stdout | 2023-08-08 05:56:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264619418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:56:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:56:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:56:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:57:00,114 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:57:00,202 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} builder stdout | 2023-08-08 05:57:00,706 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:57:00,716 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:57:00,716 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:57:00,721 [55] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 05:57:00,724 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 05:57:01,231 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:57:01,514 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:57:01,849 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:57:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:57:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:05.371682+00:00 (in 3.192661 seconds) notificationworker stdout | 2023-08-08 05:57:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:02 UTC)" (scheduled at 2023-08-08 05:57:02.178530+00:00) notificationworker stdout | 2023-08-08 05:57:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:57:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:57:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:57:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:12.293757+00:00 (in 9.999286 seconds) chunkcleanupworker stdout | 2023-08-08 05:57:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:12 UTC)" (scheduled at 2023-08-08 05:57:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:57:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:57:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 2, 294969), True, datetime.datetime(2023, 8, 8, 5, 57, 2, 294969), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:57:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:57:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:57:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:57:03,010 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:57:03,013 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:57:03,016 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:57:03,018 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:57:03,022 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:57:03,025 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:57:03,028 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:57:03,031 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:57:03,033 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 05:57:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:57:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:09.793658+00:00 (in 5.605707 seconds) repositorygcworker stdout | 2023-08-08 05:57:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:04 UTC)" (scheduled at 2023-08-08 05:57:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:57:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:57:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:57:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:57:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:15.371682+00:00 (in 9.999487 seconds) notificationworker stdout | 2023-08-08 05:57:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:15 UTC)" (scheduled at 2023-08-08 05:57:05.371682+00:00) notificationworker stdout | 2023-08-08 05:57:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:57:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 5, 372837), True, datetime.datetime(2023, 8, 8, 5, 57, 5, 372837), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:57:05,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:57:05,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:57:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:57:07,179 [197] [DEBUG] [app] Starting request: urn:request:c3dd2379-e80b-4135-9ff1-4ebd2d8dfcb2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:57:07,179 [200] [DEBUG] [app] Starting request: urn:request:a39d5fcc-1345-4a5e-94e3-bf692494e1db (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:57:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:57:07,185 [209] [DEBUG] [app] Starting request: urn:request:4c5ec862-633c-49c6-b85f-ae9c3f333f1a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:57:07,185 [209] [DEBUG] [app] Ending request: urn:request:4c5ec862-633c-49c6-b85f-ae9c3f333f1a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4c5ec862-633c-49c6-b85f-ae9c3f333f1a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:57:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:57:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:57:07,187 [205] [DEBUG] [app] Starting request: urn:request:31cc3074-544d-4160-8bbd-b46132d08d32 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2023-08-08 05:57:07,188 [205] [DEBUG] [app] Ending request: urn:request:31cc3074-544d-4160-8bbd-b46132d08d32 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:31cc3074-544d-4160-8bbd-b46132d08d32', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:57:07,189 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.005 162 0.005) gunicorn-web stdout | 2023-08-08 05:57:07,189 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:07,191 [198] [DEBUG] [app] Starting request: urn:request:4f71550a-744b-4b5a-b32a-401af9f8f6ff (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:07,191 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:07,191 [198] [DEBUG] [app] Ending request: urn:request:4f71550a-744b-4b5a-b32a-401af9f8f6ff (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4f71550a-744b-4b5a-b32a-401af9f8f6ff', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:57:07,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:57:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:57:07,193 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:07,194 [199] [DEBUG] [app] Starting request: urn:request:4a3437f2-c9a0-4f3b-8619-e05e96f80098 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:07,195 [199] [DEBUG] [app] Ending request: urn:request:4a3437f2-c9a0-4f3b-8619-e05e96f80098 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4a3437f2-c9a0-4f3b-8619-e05e96f80098', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:57:07,195 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:57:07,195 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:07,196 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:57:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:57:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:57:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:37.197412+00:00 (in 29.999449 seconds) buildlogsarchiver stdout | 2023-08-08 05:57:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:37 UTC)" (scheduled at 2023-08-08 05:57:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:57:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 57, 7, 198470), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:07,200 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:57:07,204 [197] [DEBUG] [app] Ending request: urn:request:c3dd2379-e80b-4135-9ff1-4ebd2d8dfcb2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c3dd2379-e80b-4135-9ff1-4ebd2d8dfcb2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:57:07,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:57:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:57:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 05:57:07,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:57:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:57:07,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:37 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:57:07,206 [200] [DEBUG] [app] Ending request: urn:request:a39d5fcc-1345-4a5e-94e3-bf692494e1db (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a39d5fcc-1345-4a5e-94e3-bf692494e1db', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:57:07,206 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:57:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) gunicorn-web stdout | 2023-08-08 05:57:07,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:57:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gcworker stdout | 2023-08-08 05:57:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:57:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:19.417660+00:00 (in 9.999511 seconds) gcworker stdout | 2023-08-08 05:57:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:19 UTC)" (scheduled at 2023-08-08 05:57:09.417660+00:00) gcworker stdout | 2023-08-08 05:57:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264629418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:57:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:57:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:57:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:57:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:57:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:04.187429+00:00 (in 54.393192 seconds) repositorygcworker stdout | 2023-08-08 05:57:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:09 UTC)" (scheduled at 2023-08-08 05:57:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:57:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:57:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 9, 794811), True, datetime.datetime(2023, 8, 8, 5, 57, 9, 794811), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:57:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:57:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:57:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:09 UTC)" executed successfully builder stdout | 2023-08-08 05:57:10,730 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:57:10,738 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:57:10,738 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:57:10,742 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:57:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:57:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:22.293757+00:00 (in 9.999455 seconds) chunkcleanupworker stdout | 2023-08-08 05:57:12,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:22 UTC)" (scheduled at 2023-08-08 05:57:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:57:12,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:57:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 12, 295265), True, datetime.datetime(2023, 8, 8, 5, 57, 12, 295265), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:57:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:57:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:57:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:57:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:57:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:54.284093+00:00 (in 41.186202 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:57:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:13 UTC)" (scheduled at 2023-08-08 05:57:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:57:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:57:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 13, 98248), True, datetime.datetime(2023, 8, 8, 5, 57, 13, 98248), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:57:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:57:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:57:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:57:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:57:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:25.371682+00:00 (in 9.999497 seconds) notificationworker stdout | 2023-08-08 05:57:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:25 UTC)" (scheduled at 2023-08-08 05:57:15.371682+00:00) notificationworker stdout | 2023-08-08 05:57:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:57:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 15, 372747), True, datetime.datetime(2023, 8, 8, 5, 57, 15, 372747), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:57:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:57:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:57:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:57:17,315 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:57:18,577 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:57:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:57:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:29.417660+00:00 (in 9.999491 seconds) gcworker stdout | 2023-08-08 05:57:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:29 UTC)" (scheduled at 2023-08-08 05:57:19.417660+00:00) gcworker stdout | 2023-08-08 05:57:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264639418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:57:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:57:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:57:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:29 UTC)" executed successfully securityworker stdout | 2023-08-08 05:57:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:57:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:26.890741+00:00 (in 7.202914 seconds) securityworker stdout | 2023-08-08 05:57:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:49 UTC)" (scheduled at 2023-08-08 05:57:19.687363+00:00) securityworker stdout | 2023-08-08 05:57:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:57:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:57:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:57:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:57:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:57:19,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:57:19,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:57:19,729 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 05:57:20,086 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:57:20,752 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:57:20,761 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:57:20,761 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:57:20,765 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:22,179 [197] [DEBUG] [app] Starting request: urn:request:2c0a63e0-a632-4a06-9bf3-9388ae971c24 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:57:22,179 [200] [DEBUG] [app] Starting request: urn:request:a1a85547-1aed-4baf-9a33-1013271d593f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:57:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:22,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:57:22,186 [209] [DEBUG] [app] Starting request: urn:request:c43c34de-936e-4f72-a769-c44f3827f490 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:57:22,186 [208] [DEBUG] [app] Starting request: urn:request:ed0c59c9-a5df-44d4-877e-937c08c716a9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:57:22,186 [209] [DEBUG] [app] Ending request: urn:request:c43c34de-936e-4f72-a769-c44f3827f490 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c43c34de-936e-4f72-a769-c44f3827f490', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:57:22,186 [208] [DEBUG] [app] Ending request: urn:request:ed0c59c9-a5df-44d4-877e-937c08c716a9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ed0c59c9-a5df-44d4-877e-937c08c716a9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:57:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:57:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 05:57:22,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:22,193 [199] [DEBUG] [app] Starting request: urn:request:e08b2b9e-d369-43f0-9c85-d53ea5c51082 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:22,193 [198] [DEBUG] [app] Starting request: urn:request:d18ebaa9-b887-4102-92a5-1b0580f29b6a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:22,193 [199] [DEBUG] [app] Ending request: urn:request:e08b2b9e-d369-43f0-9c85-d53ea5c51082 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e08b2b9e-d369-43f0-9c85-d53ea5c51082', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:57:22,193 [198] [DEBUG] [app] Ending request: urn:request:d18ebaa9-b887-4102-92a5-1b0580f29b6a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d18ebaa9-b887-4102-92a5-1b0580f29b6a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:57:22,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.003) gunicorn-web stdout | 2023-08-08 05:57:22,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:22,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:57:22,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:22,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:57:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:57:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:57:22,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:57:22,204 [197] [DEBUG] [app] Ending request: urn:request:2c0a63e0-a632-4a06-9bf3-9388ae971c24 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2c0a63e0-a632-4a06-9bf3-9388ae971c24', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:57:22,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:22,205 [200] [DEBUG] [app] Ending request: urn:request:a1a85547-1aed-4baf-9a33-1013271d593f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a1a85547-1aed-4baf-9a33-1013271d593f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:57:22,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:57:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:57:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:57:22,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:57:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:57:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) chunkcleanupworker stdout | 2023-08-08 05:57:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:57:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:28.874906+00:00 (in 6.580584 seconds) chunkcleanupworker stdout | 2023-08-08 05:57:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:32 UTC)" (scheduled at 2023-08-08 05:57:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:57:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:57:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 22, 294813), True, datetime.datetime(2023, 8, 8, 5, 57, 22, 294813), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:57:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:57:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:57:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:57:22,858 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:57:24,698 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 05:57:24,781 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-secscan stdout | 2023-08-08 05:57:24,795 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:57:24,800 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:57:24,803 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} chunkcleanupworker stdout | 2023-08-08 05:57:25,080 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:57:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:57:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:35.371682+00:00 (in 9.999439 seconds) notificationworker stdout | 2023-08-08 05:57:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:35 UTC)" (scheduled at 2023-08-08 05:57:25.371682+00:00) notificationworker stdout | 2023-08-08 05:57:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:57:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 25, 372898), True, datetime.datetime(2023, 8, 8, 5, 57, 25, 372898), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:57:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:57:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:57:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:57:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:57:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:47.398551+00:00 (in 21.883568 seconds) exportactionlogsworker stdout | 2023-08-08 05:57:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:25 UTC)" (scheduled at 2023-08-08 05:57:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:57:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:57:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:57:26,174 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:57:26,713 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:57:26,871 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:57:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:57:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:49.687363+00:00 (in 22.796092 seconds) securityworker stdout | 2023-08-08 05:57:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:56 UTC)" (scheduled at 2023-08-08 05:57:26.890741+00:00) securityworker stdout | 2023-08-08 05:57:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:57:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:57:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:57:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:57:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:57:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:57:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:57:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:57:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:57:26,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:57:26,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:57:26,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:57:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:57:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:57:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:57:26,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:57:26,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:57:26,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:57:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:57:27,202 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:57:27,222 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:57:27,231 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:57:27,244 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:57:27,247 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:57:28,095 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:57:28,278 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:57:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:57:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:32.293757+00:00 (in 3.418293 seconds) chunkcleanupworker stdout | 2023-08-08 05:57:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:28 UTC)" (scheduled at 2023-08-08 05:57:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:57:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:57:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:57:29,335 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:57:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:57:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:39.417660+00:00 (in 9.999473 seconds) gcworker stdout | 2023-08-08 05:57:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:39 UTC)" (scheduled at 2023-08-08 05:57:29.417660+00:00) gcworker stdout | 2023-08-08 05:57:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264649418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:57:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:57:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:57:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:57:30,151 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:57:30,209 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:57:30,734 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 05:57:30,775 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:57:30,784 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:57:30,784 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:57:30,788 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:31,226 [197] [DEBUG] [app] Starting request: urn:request:36cf3447-a687-4d88-9bed-31a69173b006 (/api/v1/repository/build_test/github/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:57:31,226 [198] [DEBUG] [app] Starting request: urn:request:82eeb58d-44ff-422a-8feb-221cbfa87e9d (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:57:31,226 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,226 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,226 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,226 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,227 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,227 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,235 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:57:31,235 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:57:31,235 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:57:31,235 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:57:31,235 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,235 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,235 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:57:31,235 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,235 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,235 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,235 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,235 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:57:31,236 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,236 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,236 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,236 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,238 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:57:31,238 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:57:31,244 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,245 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,245 [198] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2023-08-08 05:57:31,246 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,246 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,247 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [2, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,249 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [2, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:57:31,249 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,250 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,250 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,251 [198] [DEBUG] [app] Ending request: urn:request:82eeb58d-44ff-422a-8feb-221cbfa87e9d (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:82eeb58d-44ff-422a-8feb-221cbfa87e9d', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:57:31,251 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [2, None, 1691474251250, False, 101, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,252 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:31,252 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:57:31 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:57:31 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/2.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 88 0.027) gunicorn-web stdout | 2023-08-08 05:57:31,253 [197] [DEBUG] [app] Ending request: urn:request:36cf3447-a687-4d88-9bed-31a69173b006 (/api/v1/repository/build_test/github/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:36cf3447-a687-4d88-9bed-31a69173b006', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/github/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:57:31,254 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:57:31 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 886 0.030) gunicorn-web stdout | 2023-08-08 05:57:31,254 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:57:31 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityscanningnotificationworker stdout | 2023-08-08 05:57:31,268 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:57:31,546 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-web stdout | 2023-08-08 05:57:31,849 [200] [DEBUG] [app] Starting request: urn:request:03b62b52-0d38-4758-83ea-8f9605aa7499 (/api/v1/repository/build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:57:31,849 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,849 [197] [DEBUG] [app] Starting request: urn:request:08940ead-ba22-4870-9d2c-4e375ea2fab3 (/api/v1/repository/build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:57:31,849 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,850 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,850 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,850 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,851 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,857 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:57:31,857 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:57:31,857 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:57:31,857 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:57:31,857 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,857 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,858 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:57:31,858 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:57:31,858 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,858 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:31,858 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,858 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,858 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,858 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,858 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,858 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,860 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:57:31,860 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) expiredappspecifictokenworker stdout | 2023-08-08 05:57:31,863 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-web stdout | 2023-08-08 05:57:31,871 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,871 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:57:31,872 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,873 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:31,875 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'github']) gunicorn-web stdout | 2023-08-08 05:57:31,877 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'github']) gunicorn-web stdout | 2023-08-08 05:57:31,878 [200] [DEBUG] [app] Ending request: urn:request:03b62b52-0d38-4758-83ea-8f9605aa7499 (/api/v1/repository/build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:03b62b52-0d38-4758-83ea-8f9605aa7499', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/', 'path': '/api/v1/repository/build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:57:31,878 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:57:31 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/2.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.032 413 0.030) gunicorn-web stdout | 2023-08-08 05:57:31,879 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:57:31 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/1.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:57:31,880 [197] [DEBUG] [app] Ending request: urn:request:08940ead-ba22-4870-9d2c-4e375ea2fab3 (/api/v1/repository/build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:08940ead-ba22-4870-9d2c-4e375ea2fab3', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/', 'path': '/api/v1/repository/build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:57:31,881 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:57:31 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/2.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.033 68 0.033) gunicorn-web stdout | 2023-08-08 05:57:31,882 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:57:31 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/1.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:57:32,041 [199] [DEBUG] [app] Starting request: urn:request:df870378-3cb2-474b-a357-85c8cb7144ea (/repository/build_test/github/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:57:32,043 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:32,051 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:32,052 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:32,056 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['github', 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:32,062 [199] [DEBUG] [app] Ending request: urn:request:df870378-3cb2-474b-a357-85c8cb7144ea (/repository/build_test/github/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:df870378-3cb2-474b-a357-85c8cb7144ea', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f', 'path': '/repository/build_test/github/status', 'parameters': {'token': '245e22c3-ba4f-4e53-9a58-fce8fb4eb90f'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:57:32,063 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:32,063 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:57:32 +0000] "GET /repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:57:32 +0000] "GET /repository/build_test/github/status?token=245e22c3-ba4f-4e53-9a58-fce8fb4eb90f HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.024 124 0.023) gunicorn-web stdout | 2023-08-08 05:57:32,236 [197] [DEBUG] [app] Starting request: urn:request:cb6efcd7-c91f-414e-9984-7427786b70e7 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:57:32,236 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:32,236 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:32,237 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:32,244 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:57:32,244 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:57:32,244 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:32,244 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:57:32,244 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:32,244 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:32,244 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:32,245 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:32,247 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:57:32,253 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:57:32,254 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:32,258 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:57:32,261 [197] [DEBUG] [app] Ending request: urn:request:cb6efcd7-c91f-414e-9984-7427786b70e7 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:cb6efcd7-c91f-414e-9984-7427786b70e7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:57:32,262 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:32,262 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:57:32 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:57:32 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 73 0.027) chunkcleanupworker stdout | 2023-08-08 05:57:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:57:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:42.293757+00:00 (in 9.999368 seconds) chunkcleanupworker stdout | 2023-08-08 05:57:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:42 UTC)" (scheduled at 2023-08-08 05:57:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:57:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:57:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 32, 294907), True, datetime.datetime(2023, 8, 8, 5, 57, 32, 294907), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:57:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:57:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:57:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:42 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:57:32,626 [198] [DEBUG] [app] Starting request: urn:request:7d915c6a-d539-4f2d-8f14-20c4acc565f8 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:57:32,626 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:32,626 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:32,627 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:32,635 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:57:32,635 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:57:32,635 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:32,636 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:57:32,636 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:57:32,636 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:32,636 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:32,636 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:57:32,638 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:57:32,645 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:57:32,646 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:32,650 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 10]) gunicorn-web stdout | 2023-08-08 05:57:32,654 [198] [DEBUG] [app] Ending request: urn:request:7d915c6a-d539-4f2d-8f14-20c4acc565f8 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:7d915c6a-d539-4f2d-8f14-20c4acc565f8', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=10', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:57:32,654 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:57:32 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 418 0.030) gunicorn-web stdout | 2023-08-08 05:57:32,655 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:57:32 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-registry stdout | 2023-08-08 05:57:33,021 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:57:33,023 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:57:33,026 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:57:33,029 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:57:33,031 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:57:33,034 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:57:33,041 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:57:33,044 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:57:33,047 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 05:57:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:57:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:45.371682+00:00 (in 9.999419 seconds) notificationworker stdout | 2023-08-08 05:57:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:45 UTC)" (scheduled at 2023-08-08 05:57:35.371682+00:00) notificationworker stdout | 2023-08-08 05:57:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:57:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 35, 372764), True, datetime.datetime(2023, 8, 8, 5, 57, 35, 372764), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:57:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:57:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:57:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:57:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:57:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:54.393627+00:00 (in 17.813287 seconds) namespacegcworker stdout | 2023-08-08 05:57:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:36 UTC)" (scheduled at 2023-08-08 05:57:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:57:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:57:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:57:37,179 [197] [DEBUG] [app] Starting request: urn:request:43291a16-129c-4f0f-b50a-4738b3476f9a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:57:37,179 [200] [DEBUG] [app] Starting request: urn:request:a1f026e1-7707-4a26-a8ec-5f0db7b575af (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:57:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:57:37,185 [210] [DEBUG] [app] Starting request: urn:request:65da1be2-9216-4493-81f0-513040347ce3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:57:37,185 [209] [DEBUG] [app] Starting request: urn:request:416d38e5-ada4-4743-aa9b-dc6a9ac62544 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:57:37,185 [209] [DEBUG] [app] Ending request: urn:request:416d38e5-ada4-4743-aa9b-dc6a9ac62544 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:416d38e5-ada4-4743-aa9b-dc6a9ac62544', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:57:37,185 [210] [DEBUG] [app] Ending request: urn:request:65da1be2-9216-4493-81f0-513040347ce3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:65da1be2-9216-4493-81f0-513040347ce3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:57:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:57:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:57:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:37,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:37,191 [200] [DEBUG] [app] Starting request: urn:request:80df3e49-88ad-4add-b4f5-43849bb2e9f9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:37,191 [200] [DEBUG] [app] Ending request: urn:request:80df3e49-88ad-4add-b4f5-43849bb2e9f9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:80df3e49-88ad-4add-b4f5-43849bb2e9f9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:57:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:57:37,191 [199] [DEBUG] [app] Starting request: urn:request:237876ff-08e8-44a2-aa82-327602093aaf (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:37,192 [199] [DEBUG] [app] Ending request: urn:request:237876ff-08e8-44a2-aa82-327602093aaf (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:237876ff-08e8-44a2-aa82-327602093aaf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:57:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:57:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:57:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:57:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:57:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:57:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:57:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:07.197412+00:00 (in 29.999412 seconds) buildlogsarchiver stdout | 2023-08-08 05:57:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:07 UTC)" (scheduled at 2023-08-08 05:57:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:57:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 57, 37, 198442), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:57:37,202 [200] [DEBUG] [app] Ending request: urn:request:a1f026e1-7707-4a26-a8ec-5f0db7b575af (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a1f026e1-7707-4a26-a8ec-5f0db7b575af', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:57:37,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:57:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:57:37,203 [197] [DEBUG] [app] Ending request: urn:request:43291a16-129c-4f0f-b50a-4738b3476f9a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:43291a16-129c-4f0f-b50a-4738b3476f9a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:57:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 05:57:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:57:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:57:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) buildlogsarchiver stdout | 2023-08-08 05:57:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:57:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:57:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:07 UTC)" executed successfully gcworker stdout | 2023-08-08 05:57:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:57:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:49.417660+00:00 (in 9.999514 seconds) gcworker stdout | 2023-08-08 05:57:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:49 UTC)" (scheduled at 2023-08-08 05:57:39.417660+00:00) gcworker stdout | 2023-08-08 05:57:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264659418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:57:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:57:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:57:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:49 UTC)" executed successfully builder stdout | 2023-08-08 05:57:40,795 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:57:40,804 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:57:40,804 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:57:40,809 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:57:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:57:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:52.293757+00:00 (in 9.999446 seconds) chunkcleanupworker stdout | 2023-08-08 05:57:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:52 UTC)" (scheduled at 2023-08-08 05:57:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:57:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:57:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 42, 294844), True, datetime.datetime(2023, 8, 8, 5, 57, 42, 294844), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:57:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:57:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:57:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:57:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:57:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:55.371682+00:00 (in 9.999471 seconds) notificationworker stdout | 2023-08-08 05:57:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:55 UTC)" (scheduled at 2023-08-08 05:57:45.371682+00:00) notificationworker stdout | 2023-08-08 05:57:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:57:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 45, 372715), True, datetime.datetime(2023, 8, 8, 5, 57, 45, 372715), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:57:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:57:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:57:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:55 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:57:47,348 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} exportactionlogsworker stdout | 2023-08-08 05:57:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:57:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:25.514341+00:00 (in 38.115287 seconds) exportactionlogsworker stdout | 2023-08-08 05:57:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:47 UTC)" (scheduled at 2023-08-08 05:57:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:57:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:57:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 47, 399542), True, datetime.datetime(2023, 8, 8, 5, 57, 47, 399542), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:57:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:57:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:57:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:47 UTC)" executed successfully logrotateworker stdout | 2023-08-08 05:57:48,615 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:57:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:57:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:59.417660+00:00 (in 9.999495 seconds) gcworker stdout | 2023-08-08 05:57:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:59 UTC)" (scheduled at 2023-08-08 05:57:49.417660+00:00) gcworker stdout | 2023-08-08 05:57:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264669418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:57:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:57:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:57:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:57:59 UTC)" executed successfully securityworker stdout | 2023-08-08 05:57:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:57:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:57:56.890741+00:00 (in 7.202823 seconds) securityworker stdout | 2023-08-08 05:57:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:19 UTC)" (scheduled at 2023-08-08 05:57:49.687363+00:00) securityworker stdout | 2023-08-08 05:57:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:57:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:57:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:57:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:57:49,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:57:49,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:57:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:57:49,738 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 05:57:50,122 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:57:50,819 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:57:50,828 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:57:50,828 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:57:50,833 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:52,178 [197] [DEBUG] [app] Starting request: urn:request:5c33feb4-3b07-43e2-9eb4-6fe51af21569 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:57:52,178 [200] [DEBUG] [app] Starting request: urn:request:438b3a61-828e-409f-9ad7-05f1b110a3e9 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:57:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:52,182 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:57:52,183 [209] [DEBUG] [app] Starting request: urn:request:14c07204-9848-4c00-b127-bb89d4e05534 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:57:52,184 [203] [DEBUG] [app] Starting request: urn:request:2b8f24c8-c287-4732-8893-20972dab10a7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:57:52,184 [209] [DEBUG] [app] Ending request: urn:request:14c07204-9848-4c00-b127-bb89d4e05534 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:14c07204-9848-4c00-b127-bb89d4e05534', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:57:52,184 [203] [DEBUG] [app] Ending request: urn:request:2b8f24c8-c287-4732-8893-20972dab10a7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2b8f24c8-c287-4732-8893-20972dab10a7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-registry stdout | 2023-08-08 05:57:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:57:52,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:57:52,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:52,186 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:52,186 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:57:52,188 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:52,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:57:52,190 [198] [DEBUG] [app] Starting request: urn:request:401a3652-d14f-48db-86aa-39898961ed25 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:52,190 [198] [DEBUG] [app] Ending request: urn:request:401a3652-d14f-48db-86aa-39898961ed25 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:401a3652-d14f-48db-86aa-39898961ed25', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:57:52,191 [199] [DEBUG] [app] Starting request: urn:request:b87fc7ea-e095-4988-a460-5985b8288222 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:57:52,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:57:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:52,191 [199] [DEBUG] [app] Ending request: urn:request:b87fc7ea-e095-4988-a460-5985b8288222 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b87fc7ea-e095-4988-a460-5985b8288222', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:57:52,191 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:57:52,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:57:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:57:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:57:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:57:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:57:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:57:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:57:52,202 [197] [DEBUG] [app] Ending request: urn:request:5c33feb4-3b07-43e2-9eb4-6fe51af21569 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5c33feb4-3b07-43e2-9eb4-6fe51af21569', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:57:52,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:57:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:57:52,203 [200] [DEBUG] [app] Ending request: urn:request:438b3a61-828e-409f-9ad7-05f1b110a3e9 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:438b3a61-828e-409f-9ad7-05f1b110a3e9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:57:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:57:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:57:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:57:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:57:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 05:57:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:57:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:02.293757+00:00 (in 9.999485 seconds) chunkcleanupworker stdout | 2023-08-08 05:57:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:02 UTC)" (scheduled at 2023-08-08 05:57:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:57:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:57:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 52, 294767), True, datetime.datetime(2023, 8, 8, 5, 57, 52, 294767), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:57:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:57:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:57:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:57:52,892 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:57:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:57:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:13.097375+00:00 (in 18.812775 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:57:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:54 UTC)" (scheduled at 2023-08-08 05:57:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:57:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:57:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:57:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:57:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:36.579841+00:00 (in 42.185704 seconds) namespacegcworker stdout | 2023-08-08 05:57:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:54 UTC)" (scheduled at 2023-08-08 05:57:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:57:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:57:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 54, 394686), True, datetime.datetime(2023, 8, 8, 5, 57, 54, 394686), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:57:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:57:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:57:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:58:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:57:54,722 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} gunicorn-secscan stdout | 2023-08-08 05:57:54,803 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:57:54,807 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:57:54,810 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} repositorygcworker stdout | 2023-08-08 05:57:54,814 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:57:55,113 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:57:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:57:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:02.178530+00:00 (in 6.806325 seconds) notificationworker stdout | 2023-08-08 05:57:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:05 UTC)" (scheduled at 2023-08-08 05:57:55.371682+00:00) notificationworker stdout | 2023-08-08 05:57:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:57:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 57, 55, 372716), True, datetime.datetime(2023, 8, 8, 5, 57, 55, 372716), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:57:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:57:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:57:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:57:56,192 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:57:56,738 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 05:57:56,888 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 05:57:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:57:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:19.687363+00:00 (in 22.796094 seconds) securityworker stdout | 2023-08-08 05:57:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:26 UTC)" (scheduled at 2023-08-08 05:57:56.890741+00:00) securityworker stdout | 2023-08-08 05:57:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:57:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:57:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:57:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:57:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:57:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:57:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:57:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:57:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:57:56,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:57:56,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:57:56,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:57:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:57:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:57:56,904 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:57:56,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:57:56,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:57:56,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:57:57,211 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:57:57,233 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:57:57,243 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:57:57,255 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:57:57,258 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:57:58,132 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:57:58,313 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 05:57:59,374 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:57:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:57:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:09.417660+00:00 (in 9.999503 seconds) gcworker stdout | 2023-08-08 05:57:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:09 UTC)" (scheduled at 2023-08-08 05:57:59.417660+00:00) gcworker stdout | 2023-08-08 05:57:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264679418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:57:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:57:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:57:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:58:00,159 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:58:00,246 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:58:00,770 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 05:58:00,841 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:58:00,850 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:58:00,851 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:58:00,855 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:58:01,305 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:58:01,576 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:58:01,894 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:58:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:58:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:05.371682+00:00 (in 3.192578 seconds) notificationworker stdout | 2023-08-08 05:58:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:02 UTC)" (scheduled at 2023-08-08 05:58:02.178530+00:00) notificationworker stdout | 2023-08-08 05:58:02,180 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:58:02,180 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:58:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:58:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:12.293757+00:00 (in 9.999505 seconds) chunkcleanupworker stdout | 2023-08-08 05:58:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:12 UTC)" (scheduled at 2023-08-08 05:58:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:58:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:58:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 2, 294735), True, datetime.datetime(2023, 8, 8, 5, 58, 2, 294735), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:58:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:58:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:58:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:58:03,030 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:58:03,033 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:58:03,036 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:58:03,041 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:58:03,044 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:58:03,047 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:58:03,052 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:58:03,055 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:58:03,057 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-web stdout | 2023-08-08 05:58:03,473 [198] [DEBUG] [app] Starting request: urn:request:53def45d-750a-46a6-92d2-5bad742d0809 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:03,473 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:03,473 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:03,474 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:03,481 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:03,481 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:03,481 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:03,482 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:03,482 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:03,482 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:03,482 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:03,482 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:03,484 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:03,490 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:03,491 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:03,495 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:58:03,499 [198] [DEBUG] [app] Ending request: urn:request:53def45d-750a-46a6-92d2-5bad742d0809 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:53def45d-750a-46a6-92d2-5bad742d0809', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:03,499 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:03 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 867 0.028) gunicorn-web stdout | 2023-08-08 05:58:03,500 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:03 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" repositorygcworker stdout | 2023-08-08 05:58:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:58:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:09.793658+00:00 (in 5.605636 seconds) repositorygcworker stdout | 2023-08-08 05:58:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:04 UTC)" (scheduled at 2023-08-08 05:58:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:58:04,189 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:58:04,189 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:58:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:58:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:15.371682+00:00 (in 9.999431 seconds) notificationworker stdout | 2023-08-08 05:58:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:15 UTC)" (scheduled at 2023-08-08 05:58:05.371682+00:00) notificationworker stdout | 2023-08-08 05:58:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:58:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 5, 372763), True, datetime.datetime(2023, 8, 8, 5, 58, 5, 372763), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:58:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:58:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:58:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:58:07,178 [197] [DEBUG] [app] Starting request: urn:request:40c48bb5-c0ed-4f10-9257-62ecf6689c0b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:58:07,179 [200] [DEBUG] [app] Starting request: urn:request:93a4adb3-a223-4a97-8698-54df7d9319fd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:58:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:07,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:58:07,184 [203] [DEBUG] [app] Starting request: urn:request:d2433151-8f63-4768-ad1e-a75d8c5c318f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:58:07,185 [208] [DEBUG] [app] Starting request: urn:request:a8560b72-b176-4005-8252-2de4ed392a6e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:58:07,185 [203] [DEBUG] [app] Ending request: urn:request:d2433151-8f63-4768-ad1e-a75d8c5c318f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d2433151-8f63-4768-ad1e-a75d8c5c318f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 05:58:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:58:07,185 [208] [DEBUG] [app] Ending request: urn:request:a8560b72-b176-4005-8252-2de4ed392a6e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a8560b72-b176-4005-8252-2de4ed392a6e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:58:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:58:07,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:58:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:07,190 [199] [DEBUG] [app] Starting request: urn:request:12f9e5b8-917d-473b-ba8d-32e42b31fed8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:58:07,191 [197] [DEBUG] [app] Starting request: urn:request:a8cb4d5d-8a4e-4ee7-aa91-6e3ec4f4d322 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:58:07,191 [199] [DEBUG] [app] Ending request: urn:request:12f9e5b8-917d-473b-ba8d-32e42b31fed8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:12f9e5b8-917d-473b-ba8d-32e42b31fed8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:58:07,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:58:07,191 [197] [DEBUG] [app] Ending request: urn:request:a8cb4d5d-8a4e-4ee7-aa91-6e3ec4f4d322 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a8cb4d5d-8a4e-4ee7-aa91-6e3ec4f4d322', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:58:07,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:58:07,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:58:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:58:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 05:58:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:58:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:37.197412+00:00 (in 29.999547 seconds) gunicorn-web stdout | 2023-08-08 05:58:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:58:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:37 UTC)" (scheduled at 2023-08-08 05:58:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:58:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 58, 7, 198287), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:07,202 [197] [DEBUG] [app] Ending request: urn:request:40c48bb5-c0ed-4f10-9257-62ecf6689c0b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:40c48bb5-c0ed-4f10-9257-62ecf6689c0b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:58:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:58:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:58:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 05:58:07,204 [200] [DEBUG] [app] Ending request: urn:request:93a4adb3-a223-4a97-8698-54df7d9319fd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:93a4adb3-a223-4a97-8698-54df7d9319fd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:58:07,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:58:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:58:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 05:58:07,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:58:07,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:58:07,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:37 UTC)" executed successfully gcworker stdout | 2023-08-08 05:58:09,417 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:58:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:19.417660+00:00 (in 9.999552 seconds) gcworker stdout | 2023-08-08 05:58:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:19 UTC)" (scheduled at 2023-08-08 05:58:09.417660+00:00) gcworker stdout | 2023-08-08 05:58:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264689418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:58:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:58:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:58:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:58:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:58:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:04.187429+00:00 (in 54.393253 seconds) repositorygcworker stdout | 2023-08-08 05:58:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:09 UTC)" (scheduled at 2023-08-08 05:58:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:58:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:58:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 9, 794773), True, datetime.datetime(2023, 8, 8, 5, 58, 9, 794773), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:58:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:58:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:58:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:09 UTC)" executed successfully builder stdout | 2023-08-08 05:58:10,860 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:58:10,869 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:58:10,870 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:58:10,874 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:58:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:58:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:22.293757+00:00 (in 9.999502 seconds) chunkcleanupworker stdout | 2023-08-08 05:58:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:22 UTC)" (scheduled at 2023-08-08 05:58:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:58:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:58:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 12, 294851), True, datetime.datetime(2023, 8, 8, 5, 58, 12, 294851), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:58:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:58:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:58:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:58:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:58:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:54.284093+00:00 (in 41.186177 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:58:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:13 UTC)" (scheduled at 2023-08-08 05:58:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:58:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:58:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 13, 98275), True, datetime.datetime(2023, 8, 8, 5, 58, 13, 98275), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:58:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:58:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:58:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:58:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:58:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:25.371682+00:00 (in 9.999475 seconds) notificationworker stdout | 2023-08-08 05:58:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:25 UTC)" (scheduled at 2023-08-08 05:58:15.371682+00:00) notificationworker stdout | 2023-08-08 05:58:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:58:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 15, 372702), True, datetime.datetime(2023, 8, 8, 5, 58, 15, 372702), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:58:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:58:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:58:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:58:17,380 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gunicorn-web stdout | 2023-08-08 05:58:18,430 [197] [DEBUG] [app] Starting request: urn:request:c036d91e-8750-4579-81b1-d949644e25c4 (/api/v1/repository/build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:18,430 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:18,430 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:18,431 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:18,438 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:18,438 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:18,438 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:18,439 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:18,439 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:18,439 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:18,439 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:18,439 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:18,441 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:18,448 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:18,449 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:18,452 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'github']) gunicorn-web stdout | 2023-08-08 05:58:18,454 [197] [DEBUG] [app] Ending request: urn:request:c036d91e-8750-4579-81b1-d949644e25c4 (/api/v1/repository/build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:c036d91e-8750-4579-81b1-d949644e25c4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/', 'path': '/api/v1/repository/build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:18,455 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:18,456 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:18 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/1.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:18 +0000] "GET /api/v1/repository/build_test/github/trigger/ HTTP/2.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 861 0.027) logrotateworker stdout | 2023-08-08 05:58:18,623 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:58:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:58:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:29.417660+00:00 (in 9.999513 seconds) gcworker stdout | 2023-08-08 05:58:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:29 UTC)" (scheduled at 2023-08-08 05:58:19.417660+00:00) gcworker stdout | 2023-08-08 05:58:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264699418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:58:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:58:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:58:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:29 UTC)" executed successfully securityworker stdout | 2023-08-08 05:58:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:58:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:26.890741+00:00 (in 7.202847 seconds) securityworker stdout | 2023-08-08 05:58:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:49 UTC)" (scheduled at 2023-08-08 05:58:19.687363+00:00) securityworker stdout | 2023-08-08 05:58:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:58:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:58:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:58:19,691 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:58:19,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:58:19,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:58:19,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:58:19,761 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 05:58:20,156 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:58:20,884 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:58:20,896 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:58:20,896 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:58:20,900 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:21,251 [197] [DEBUG] [app] Starting request: urn:request:878c73c7-3cf1-4e0a-8b7b-aa738d88d836 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:21,252 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:21,252 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:21,252 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:21,258 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:21,258 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:21,258 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:21,259 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:21,259 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:21,259 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:21,259 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:21,259 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:21,261 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:21,267 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:21,268 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:21,271 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:21,273 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:21,276 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:21,278 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:58:21,668 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:58:21,669 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:58:21 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4952', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '48', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '995A:26FE:100FA6E:20CB89D:64D1D97D'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAVJU7YTPGHBHV4JBYTE2HNKS","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:58:21,959 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:58:21,960 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:58:21 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"4cfa10a4c34b3fc4519e3ef97204ef558181d88187b3aea2a28f8cff9e905938"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4951', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '49', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '995A:26FE:100FABB:20CB950:64D1D97D'} [{"name":"branch1","commit":{"sha":"1d34effde035584623f29666b8f41b42e020ad4a","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/1d34effde035584623f29666b8f41b42e020ad4a"},"protected":false},{"name":"branch2","commit":{"sha":"c6ef6983d9344c211af3fc7312bddccbe7c0dd7c","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/c6ef6983d9344c211af3fc7312bddccbe7c0dd7c"},"protected":false},{"name":"hu-weihua-patch-1","commit":{"sha":"7f19019c0b0bf7eb48b55479771b213e8081c884","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/7f19019c0b0bf7eb48b55479771b213e8081c884"},"protected":false},{"name":"hu-weihua-patch-2","commit":{"sha":"65e01fcd85dfe53a2d52b5f7e188fca3f30daf78","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/65e01fcd85dfe53a2d52b5f7e188fca3f30daf78"},"protected":false},{"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d"},"protected":false}] gunicorn-web stdout | 2023-08-08 05:58:21,964 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 05:58:22,179 [197] [DEBUG] [app] Starting request: urn:request:1f2d7e8b-515e-4d53-b075-68c0afd9dc1d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:58:22,179 [200] [DEBUG] [app] Starting request: urn:request:1a14b608-2b74-40d9-aaf8-009713097ee1 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:58:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:58:22,185 [205] [DEBUG] [app] Starting request: urn:request:14f36c41-6ebc-4cc0-af9b-c4345e403e66 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:58:22,185 [203] [DEBUG] [app] Starting request: urn:request:efc561b1-db66-4632-b016-039f21e83da8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:58:22,185 [205] [DEBUG] [app] Ending request: urn:request:14f36c41-6ebc-4cc0-af9b-c4345e403e66 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:14f36c41-6ebc-4cc0-af9b-c4345e403e66', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:58:22,185 [203] [DEBUG] [app] Ending request: urn:request:efc561b1-db66-4632-b016-039f21e83da8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:efc561b1-db66-4632-b016-039f21e83da8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:58:22,185 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 05:58:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:58:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:58:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:22,191 [197] [DEBUG] [app] Starting request: urn:request:bec4797c-a767-46c1-b83f-d6d7b6b320fd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:58:22,191 [199] [DEBUG] [app] Starting request: urn:request:569fee1a-0cf0-4331-bd20-6c9fda75c52f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:58:22,192 [197] [DEBUG] [app] Ending request: urn:request:bec4797c-a767-46c1-b83f-d6d7b6b320fd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bec4797c-a767-46c1-b83f-d6d7b6b320fd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:58:22,192 [199] [DEBUG] [app] Ending request: urn:request:569fee1a-0cf0-4331-bd20-6c9fda75c52f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:569fee1a-0cf0-4331-bd20-6c9fda75c52f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:58:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:58:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:58:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:22,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:58:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:58:22,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:58:22,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:58:22,204 [197] [DEBUG] [app] Ending request: urn:request:1f2d7e8b-515e-4d53-b075-68c0afd9dc1d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1f2d7e8b-515e-4d53-b075-68c0afd9dc1d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:58:22,204 [200] [DEBUG] [app] Ending request: urn:request:1a14b608-2b74-40d9-aaf8-009713097ee1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1a14b608-2b74-40d9-aaf8-009713097ee1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:58:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:22,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:58:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:58:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:58:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:58:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:58:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 05:58:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:58:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:28.874906+00:00 (in 6.580643 seconds) chunkcleanupworker stdout | 2023-08-08 05:58:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:32 UTC)" (scheduled at 2023-08-08 05:58:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:58:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:58:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 22, 294754), True, datetime.datetime(2023, 8, 8, 5, 58, 22, 294754), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:58:22,354 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:58:22,354 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:58:22,355 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:58:22,929 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-web stdout | 2023-08-08 05:58:22,964 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:58:22,965 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:58:22 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4950', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '50', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'C177:57B8:11FC890:249CDC5:64D1D97D'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPASHLPVERAZFQ7PVZBDE2HNKU","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:58:23,025 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/tags HTTP/1.1" 200 2 gunicorn-web stdout | 2023-08-08 05:58:23,026 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/tags {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:58:23 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '2', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"4c75d91aaf88ebe4a7429173cbe96e13732080c64455c9da24aeaca752887ab8"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4949', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '51', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': 'C177:57B8:11FC991:249CFA7:64D1D97E'} [] gunicorn-web stdout | 2023-08-08 05:58:23,028 [197] [DEBUG] [app] Ending request: urn:request:878c73c7-3cf1-4e0a-8b7b-aa738d88d836 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs) {'endpoint': 'api.buildtriggerfieldvalues', 'request_id': 'urn:request:878c73c7-3cf1-4e0a-8b7b-aa738d88d836', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs', 'parameters': {}, 'json_body': {}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:23,029 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:23 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs HTTP/2.0" 200 225 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.780 543 1.780) gunicorn-web stdout | 2023-08-08 05:58:23,030 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:23 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/fields/refs HTTP/1.0" 200 225 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" blobuploadcleanupworker stdout | 2023-08-08 05:58:24,759 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} gunicorn-secscan stdout | 2023-08-08 05:58:24,812 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:58:24,817 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:58:24,820 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} repositorygcworker stdout | 2023-08-08 05:58:24,850 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:58:25,137 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:58:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:58:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:35.371682+00:00 (in 9.999463 seconds) notificationworker stdout | 2023-08-08 05:58:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:35 UTC)" (scheduled at 2023-08-08 05:58:25.371682+00:00) notificationworker stdout | 2023-08-08 05:58:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:58:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 25, 372836), True, datetime.datetime(2023, 8, 8, 5, 58, 25, 372836), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:58:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:58:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:58:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:35 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:58:25,409 [199] [DEBUG] [app] Starting request: urn:request:72a42b86-71f0-475e-bc01-4832cd104415 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/start) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:25,410 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:25,410 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:25,410 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,417 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:25,417 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:25,417 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:25,417 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:25,418 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:25,418 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:25,418 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:25,418 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:25,422 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:25,429 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:25,430 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,434 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,439 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d8c06fe-e700-4fee-91c0-e0e170c77f8b', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,443 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,447 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 exportactionlogsworker stdout | 2023-08-08 05:58:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:58:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:33.514094+00:00 (in 7.999233 seconds) exportactionlogsworker stdout | 2023-08-08 05:58:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:25 UTC)" (scheduled at 2023-08-08 05:58:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:58:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:58:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:58:25,589 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:58:25,590 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:58:25 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4948', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '52', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '81F0:3DFB:74F072:F7B5F6:64D1D981'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAT4UZF7WFXB6CN67QLE2HNK2","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 05:58:25,694 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches/main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:58:25,695 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches/main {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:58:25 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"d8accd6c537e1434f53369c16792470386400303ebf09d6aff58ab4a24589a86"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4947', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '53', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '81F0:3DFB:74F092:F7B637:64D1D981'} {"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","node_id":"C_kwDOG6AIe9oAKDg4MGZiNzAwZjhiNTEwM2U3YWRjMjBmMDgwMzg0MzRlOWY3MTJjNWQ","commit":{"author":{"name":"Hu Weihua","email":"hu_weihua@163.com","date":"2023-08-08T02:50:20Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2023-08-08T02:50:20Z"},"message":"Update update_commit.sh","tree":{"sha":"c5abbb94e6edfb389f732ea578c71d6db692406d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/c5abbb94e6edfb389f732ea578c71d6db692406d"},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsBcBAABCAAQBQJk0a1sCRBK7hj4Ov3rIwAA5MMIABciqM6Y7qngb0Joeh7FSIAp\nnv0pckL5M+0g6IBcVfxG4RC6Hfti5Gs1lYKBppBJMF5OUVdITpe2GOii6WcGieL9\nbo/Acx0X4liVp4QzoQQTNYRmCZFODrlb/FVuVcdHRtBO/hXNAsqmPOHGC1Guf5lG\nHWpLPZF9e8lnbI1/MIiu4O5+OK+qRw26pEbRHjukBa921OQ/27s+MATflSKJ+bBJ\nBshN6stBgyAA4kDsPwl7iIHn1xSGhADLw3q8BnDRIT5jRHzDoqHdck9VpVatNxxL\nK+9HZiaHhyQ3+CsvT5/NNMy5cWoqJvSpjNobgmw4bYg70AB2xUHy85jPOl7Rizk=\n=DSm6\n-----END PGP SIGNATURE-----\n","payload":"tree c5abbb94e6edfb389f732ea578c71d6db692406d\nparent f0030ac4a54f1c210daf84729cfd233407d43d14\nauthor Hu Weihua 1691463020 +0800\ncommitter GitHub 1691463020 +0800\n\nUpdate update_commit.sh"}},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","html_url":"https://github.com/hu-weihua/quay_build_test/commit/880fb700f8b5103e7adc20f08038434e9f712c5d","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d/comments","author":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","site_admin":false},"parents":[{"sha":"f0030ac4a54f1c210daf84729cfd233407d43d14","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/f0030ac4a54f1c210daf84729cfd233407d43d14","html_url":"https://github.com/hu-weihua/quay_build_test/commit/f0030ac4a54f1c210daf84729cfd233407d43d14"}]},"_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/branches/main","html":"https://github.com/hu-weihua/quay_build_test/tree/main"},"protected":false,"protection":{"enabled":false,"required_status_checks":{"enforcement_level":"off","contexts":[],"checks":[]}},"protection_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches/main/protection"} gunicorn-web stdout | 2023-08-08 05:58:25,854 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 05:58:25,854 [199] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 05:58:25 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"6d46118b8bdca7a35baac6c885f62f1daa976903ebacc78dd96eeabe29ccface"', 'last-modified': 'Tue, 08 Aug 2023 02:50:20 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4946', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '54', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '81F0:3DFB:74F0BC:F7B696:64D1D981'} {"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","node_id":"C_kwDOG6AIe9oAKDg4MGZiNzAwZjhiNTEwM2U3YWRjMjBmMDgwMzg0MzRlOWY3MTJjNWQ","commit":{"author":{"name":"Hu Weihua","email":"hu_weihua@163.com","date":"2023-08-08T02:50:20Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2023-08-08T02:50:20Z"},"message":"Update update_commit.sh","tree":{"sha":"c5abbb94e6edfb389f732ea578c71d6db692406d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/c5abbb94e6edfb389f732ea578c71d6db692406d"},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsBcBAABCAAQBQJk0a1sCRBK7hj4Ov3rIwAA5MMIABciqM6Y7qngb0Joeh7FSIAp\nnv0pckL5M+0g6IBcVfxG4RC6Hfti5Gs1lYKBppBJMF5OUVdITpe2GOii6WcGieL9\nbo/Acx0X4liVp4QzoQQTNYRmCZFODrlb/FVuVcdHRtBO/hXNAsqmPOHGC1Guf5lG\nHWpLPZF9e8lnbI1/MIiu4O5+OK+qRw26pEbRHjukBa921OQ/27s+MATflSKJ+bBJ\nBshN6stBgyAA4kDsPwl7iIHn1xSGhADLw3q8BnDRIT5jRHzDoqHdck9VpVatNxxL\nK+9HZiaHhyQ3+CsvT5/NNMy5cWoqJvSpjNobgmw4bYg70AB2xUHy85jPOl7Rizk=\n=DSm6\n-----END PGP SIGNATURE-----\n","payload":"tree c5abbb94e6edfb389f732ea578c71d6db692406d\nparent f0030ac4a54f1c210daf84729cfd233407d43d14\nauthor Hu Weihua 1691463020 +0800\ncommitter GitHub 1691463020 +0800\n\nUpdate update_commit.sh"}},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","html_url":"https://github.com/hu-weihua/quay_build_test/commit/880fb700f8b5103e7adc20f08038434e9f712c5d","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d/comments","author":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","site_admin":false},"parents":[{"sha":"f0030ac4a54f1c210daf84729cfd233407d43d14","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/f0030ac4a54f1c210daf84729cfd233407d43d14","html_url":"https://github.com/hu-weihua/quay_build_test/commit/f0030ac4a54f1c210daf84729cfd233407d43d14"}],"stats":{"total":1,"additions":1,"deletions":0},"files":[{"sha":"8ac65cd6da3c09ee9b225fddc5eea5023f65c6aa","filename":"update_commit.sh","status":"modified","additions":1,"deletions":0,"changes":1,"blob_url":"https://github.com/hu-weihua/quay_build_test/blob/880fb700f8b5103e7adc20f08038434e9f712c5d/update_commit.sh","raw_url":"https://github.com/hu-weihua/quay_build_test/raw/880fb700f8b5103e7adc20f08038434e9f712c5d/update_commit.sh","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/update_commit.sh?ref=880fb700f8b5103e7adc20f08038434e9f712c5d","patch":"@@ -4,3 +4,4 @@ echo \"add commit\"\n \n sep 30. 2022\n Aug 3, 2023\n+Aug 8, 2023"}]} gunicorn-web stdout | 2023-08-08 05:58:25,858 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,859 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['write', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,860 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "accesstokenkind" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['build-worker', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,862 [199] [DEBUG] [peewee] ('INSERT INTO "accesstoken" ("friendly_name", "token_name", "token_code", "repository_id", "created", "role_id", "temporary", "kind_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "accesstoken"."id"', ['Repository Build Token', 'FADSZB27FYRXH6XPTS8U1U7LR59ETDTX', 'v0$$F+G32jHCHkH2ewgpnYCOdQIjEpyL80b8t5bbkXuKK/lAJA92MVsXlpoX7xPVtPZqvC6iskJCwJ4G71zCsg==', 2, datetime.datetime(2023, 8, 8, 5, 58, 25, 861940), 2, True, 1]) gunicorn-web stdout | 2023-08-08 05:58:25,865 [199] [DEBUG] [endpoints.building] Creating build 880fb70 with repo quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/build_test/github tags ['main', 'latest'] gunicorn-web stdout | 2023-08-08 05:58:25,866 [199] [DEBUG] [peewee] ('INSERT INTO "repositorybuild" ("uuid", "repository_id", "access_token_id", "resource_key", "job_config", "phase", "started", "display_name", "trigger_id", "pull_robot_id", "logs_archived") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repositorybuild"."id"', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 2, 4, None, '{"docker_tags": ["main", "latest"], "registry": "quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com", "build_subdir": "/Dockerfile", "context": "/", "trigger_metadata": {"commit": "880fb700f8b5103e7adc20f08038434e9f712c5d", "ref": "refs/heads/main", "default_branch": "main", "git_url": "git@github.com:hu-weihua/quay_build_test.git", "commit_info": {"url": "https://github.com/hu-weihua/quay_build_test/commit/880fb700f8b5103e7adc20f08038434e9f712c5d", "message": "Update update_commit.sh", "date": "Tue, 08 Aug 2023 02:50:20 GMT", "author": {"username": "hu-weihua", "avatar_url": "https://avatars.githubusercontent.com/u/7370626?v=4", "url": "https://github.com/hu-weihua"}, "committer": {"username": "web-flow", "avatar_url": "https://avatars.githubusercontent.com/u/19864447?v=4", "url": "https://github.com/web-flow"}}}, "is_manual": true, "manual_user": "whuquay", "archive_url": null}', 'waiting', datetime.datetime(2023, 8, 8, 5, 58, 25, 866150), '880fb70', 3, None, False]) gunicorn-web stdout | 2023-08-08 05:58:25,869 [199] [DEBUG] [peewee] ('INSERT INTO "queueitem" ("queue_name", "body", "available_after", "available", "retries_remaining", "state_id") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "queueitem"."id"', ['dockerfilebuild/build_test/github/', '{"build_uuid": "d5b40e12-f92c-42e5-9c25-9e76946bc5a3", "pull_credentials": null}', datetime.datetime(2023, 8, 8, 5, 58, 25, 868735), True, 3, '36b23cdb-a78a-49b7-8672-eae09371defd']) gunicorn-web stdout | 2023-08-08 05:58:25,870 [199] [DEBUG] [peewee] ('UPDATE "repositorybuild" SET "uuid" = %s, "repository_id" = %s, "access_token_id" = %s, "resource_key" = %s, "job_config" = %s, "phase" = %s, "started" = %s, "display_name" = %s, "trigger_id" = %s, "pull_robot_id" = %s, "logs_archived" = %s, "queue_id" = %s WHERE ("repositorybuild"."id" = %s)', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 2, 4, None, '{"docker_tags": ["main", "latest"], "registry": "quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com", "build_subdir": "/Dockerfile", "context": "/", "trigger_metadata": {"commit": "880fb700f8b5103e7adc20f08038434e9f712c5d", "ref": "refs/heads/main", "default_branch": "main", "git_url": "git@github.com:hu-weihua/quay_build_test.git", "commit_info": {"url": "https://github.com/hu-weihua/quay_build_test/commit/880fb700f8b5103e7adc20f08038434e9f712c5d", "message": "Update update_commit.sh", "date": "Tue, 08 Aug 2023 02:50:20 GMT", "author": {"username": "hu-weihua", "avatar_url": "https://avatars.githubusercontent.com/u/7370626?v=4", "url": "https://github.com/hu-weihua"}, "committer": {"username": "web-flow", "avatar_url": "https://avatars.githubusercontent.com/u/19864447?v=4", "url": "https://github.com/web-flow"}}}, "is_manual": true, "manual_user": "whuquay", "archive_url": null}', 'waiting', datetime.datetime(2023, 8, 8, 5, 58, 25, 866150), '880fb70', 3, None, False, '2', 1]) gunicorn-web stdout | 2023-08-08 05:58:25,873 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:25,874 [199] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [9, 2, 1, 2, datetime.datetime(2023, 8, 8, 5, 58, 25, 872901), '10.130.2.1', '{"build_id": "d5b40e12-f92c-42e5-9c25-9e76946bc5a3", "docker_tags": ["main", "latest"], "repo": "github", "namespace": "build_test", "is_manual": true, "manual_user": "whuquay", "trigger_id": "3d8c06fe-e700-4fee-91c0-e0e170c77f8b", "trigger_kind": "github", "trigger_metadata": {"commit": "880fb700f8b5103e7adc20f08038434e9f712c5d", "ref": "refs/heads/main", "default_branch": "main", "git_url": "git@github.com:hu-weihua/quay_build_test.git", "commit_info": {"url": "https://github.com/hu-weihua/quay_build_test/commit/880fb700f8b5103e7adc20f08038434e9f712c5d", "message": "Update update_commit.sh", "date": "Tue, 08 Aug 2023 02:50:20 GMT", "author": {"username": "hu-weihua", "avatar_url": "https://avatars.githubusercontent.com/u/7370626?v=4", "url": "https://github.com/hu-weihua"}, "committer": {"username": "web-flow", "avatar_url": "https://avatars.githubusercontent.com/u/19864447?v=4", "url": "https://github.com/web-flow"}}}, "trigger_manual": true}']) gunicorn-web stdout | 2023-08-08 05:58:25,878 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."event_id", "t1"."method_id", "t1"."title", "t1"."config_json", "t1"."event_config_json", "t1"."number_of_failures", "t2"."id", "t2"."namespace_user_id", "t2"."name", "t2"."visibility_id", "t2"."description", "t2"."badge_token", "t2"."kind_id", "t2"."trust_enabled", "t2"."state", "t3"."id", "t3"."uuid", "t3"."username", "t3"."password_hash", "t3"."email", "t3"."verified", "t3"."stripe_id", "t3"."organization", "t3"."robot", "t3"."invoice_email", "t3"."invalid_login_attempts", "t3"."last_invalid_login", "t3"."removed_tag_expiration_s", "t3"."enabled", "t3"."invoice_email_address", "t3"."given_name", "t3"."family_name", "t3"."company", "t3"."location", "t3"."maximum_queued_builds_count", "t3"."creation_date", "t3"."last_accessed" FROM "repositorynotification" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") INNER JOIN "externalnotificationevent" AS "t4" ON ("t1"."event_id" = "t4"."id") WHERE ((("t3"."username" = %s) AND ("t2"."name" = %s)) AND ("t4"."name" = %s))', ['build_test', 'github', 'build_queued']) gunicorn-web stdout | 2023-08-08 05:58:25,881 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:25,881 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:25,881 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:25,882 [199] [DEBUG] [app] Ending request: urn:request:72a42b86-71f0-475e-bc01-4832cd104415 (/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/start) {'endpoint': 'api.activatebuildtrigger', 'request_id': 'urn:request:72a42b86-71f0-475e-bc01-4832cd104415', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/start', 'path': '/api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/start', 'parameters': {}, 'json_body': {'refs': {'kind': 'branch', 'name': 'main'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:25,883 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:25,884 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:25 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/start HTTP/1.0" 201 2081 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:25 +0000] "POST /api/v1/repository/build_test/github/trigger/3d8c06fe-e700-4fee-91c0-e0e170c77f8b/start HTTP/2.0" 201 2081 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.476 489 0.476) teamsyncworker stdout | 2023-08-08 05:58:26,227 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:58:26,757 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} securityworker stdout | 2023-08-08 05:58:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:58:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:49.687363+00:00 (in 22.796122 seconds) securityworker stdout | 2023-08-08 05:58:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:56 UTC)" (scheduled at 2023-08-08 05:58:26.890741+00:00) securityworker stdout | 2023-08-08 05:58:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:58:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:58:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:58:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:58:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:58:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:58:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:58:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:58:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:58:26,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:58:26,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:58:26,904 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:58:26,904 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:58:26,904 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:58:26,904 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:58:26,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:58:26,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:58:26,905 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:58:56 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 05:58:26,905 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 05:58:27,220 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:58:27,245 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:58:27,254 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:58:27,267 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:58:27,271 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:58:28,168 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:58:28,330 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:58:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:58:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:32.293757+00:00 (in 3.418331 seconds) chunkcleanupworker stdout | 2023-08-08 05:58:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:28 UTC)" (scheduled at 2023-08-08 05:58:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:58:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:58:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:28 UTC)" executed successfully securityworker stdout | 2023-08-08 05:58:29,412 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:58:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:58:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:39.417660+00:00 (in 9.999513 seconds) gcworker stdout | 2023-08-08 05:58:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:39 UTC)" (scheduled at 2023-08-08 05:58:29.417660+00:00) gcworker stdout | 2023-08-08 05:58:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264709418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:58:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:58:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:58:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:39 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:58:30,194 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:58:30,283 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:58:30,808 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 05:58:30,910 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:58:30,924 [55] [DEBUG] [buildman.manager.ephemeral] Creating build job for build d5b40e12-f92c-42e5-9c25-9e76946bc5a3 builder stdout | 2023-08-08 05:58:30,926 [55] [DEBUG] [buildman.orchestrator] publishing event: {"event": 1, "key": "buildman/production/building/d5b40e12-f92c-42e5-9c25-9e76946bc5a3", "value": "{\"job_queue_item\": {\"id\": 2, \"body\": \"{\\\"build_uuid\\\": \\\"d5b40e12-f92c-42e5-9c25-9e76946bc5a3\\\", \\\"pull_credentials\\\": null}\", \"retries_remaining\": 2}, \"max_expiration\": 1691481510, \"last_heartbeat\": null}"} builder stdout | 2023-08-08 05:58:30,926 [55] [DEBUG] [buildman.manager.ephemeral] Scheduling build job buildman/production/building/d5b40e12-f92c-42e5-9c25-9e76946bc5a3 builder stdout | 2023-08-08 05:58:30,926 [55] [DEBUG] [buildman.manager.ephemeral] Scheduling build d5b40e12-f92c-42e5-9c25-9e76946bc5a3 builder stdout | 2023-08-08 05:58:30,926 [55] [DEBUG] [buildman.manager.executor] Using service account token for Kubernetes authentication builder stdout | 2023-08-08 05:58:30,926 [55] [DEBUG] [buildman.manager.executor] Executor config: {'BUILDER_NAMESPACE': 'builder', 'BUILDER_VM_CONTAINER_IMAGE': 'brew.registry.redhat.io/rh-osbs/quay-quay-builder-qemu-rhcos-rhel8:v3.8.11', 'CONTAINER_CPU_LIMITS': '1000m', 'CONTAINER_CPU_REQUEST': '500m', 'CONTAINER_MEMORY_LIMITS': '5120Mi', 'CONTAINER_MEMORY_REQUEST': '3968Mi', 'CONTAINER_RUNTIME': 'podman', 'DEBUG': True, 'EXECUTOR': 'kubernetes', 'K8S_API_SERVER': 'api.whu412bm02.qe.devcluster.openshift.com:6443', 'K8S_API_TLS_CA': '/conf/stack/extra_ca_certs/build_cluster.crt', 'KUBERNETES_DISTRIBUTION': 'openshift', 'MINIMUM_RETRY_THRESHOLD': 0, 'NODE_SELECTOR_LABEL_KEY': 'type', 'NODE_SELECTOR_LABEL_VALUE': 'builder', 'QUAY_PASSWORD': 'eyJhbGciOiJSUzUxMiJ9.eyJzdWIiOiI3YTE0OWIwNWZkZTI0ZWMwYWQ3ZTIwMjQwZmMyYmU2MCJ9.W-4araN4Cus25Nl39ZOp5MmTZBzeJQlGVhaxh1bRWX2hcAhCaoflBf4l2U71ewD16F9My20HPSSpqIJJ1zydnrXKA731WRknwHYetVZ929_vVNUtHcbCvDYTzCseg781DihDbiWm2pK6mbUKLUEXJXzVaAo7hTbXQsgs63EpKGF3bJyN0M76ZnU8BNkMkaj_cWN0ir8As1vf_OEJudIiSmxxUruF-VMANbTkE50UlP-oh2VCqypBxQ81R5VLu4YsdXfNUwl5jdl14T9MGK0OTw7WAt5VdHBQqr3lsjd4WDWJqigemyrdYM5BC1XZXh8j8v_CfU-yaKXNugTtr8LYk1n6Wso1ThxL9eDUfYFvLRYLoJYizGfOjPcoPJugRIh-tS6ki7zy33-Ya4pkSLLX-OdN4Vq1UJvfE13X8PAOSAbWKF7-7dk1DG2LNW1M2UtEelmyVg6KQQ7cRlmMXp7QouaKO9vDKmudDH_2H9ZVrsXhSFTNmoS_YVkDbxqTs6ugtx5TLxbKYYdNnUygDVKprNdRZSZc-adH4JPuRBroA6CTup13PyW92EtePRN6G2IpXInMC9kkApur5qUynddIscb5KnCaYuKNeqQBCQDI7YxP6CCAfm1obf1nUWbDc2uk4DSNitRZJ84hsLBCLTABTyR0aCGYszCeU4zQ-rn5cc8', 'QUAY_USERNAME': '|76db245a-0d75-11e7-a544-28d244ea5a6d.j9is.751956', 'SERVICE_ACCOUNT_NAME': 'quay-builder', 'SERVICE_ACCOUNT_TOKEN': 'eyJhbGciOiJSUzI1NiIsImtpZCI6IkkwS1RXc3dNZ3ZLOFlZUE1pWFMwSDRGUFNZRkx3Tk5abEEtRlNiLTJsWlkifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjIl0sImV4cCI6MTY5MTU1MTE0OSwiaWF0IjoxNjkxNDY0NzQ5LCJpc3MiOiJodHRwczovL2t1YmVybmV0ZXMuZGVmYXVsdC5zdmMiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImJ1aWxkZXIiLCJzZXJ2aWNlYWNjb3VudCI6eyJuYW1lIjoicXVheS1idWlsZGVyIiwidWlkIjoiOThkMGRmNGYtMGRiMS00M2VkLWExMmYtY2Q4MDVjZDRiMzg3In19LCJuYmYiOjE2OTE0NjQ3NDksInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpidWlsZGVyOnF1YXktYnVpbGRlciJ9.LpmO7SvKJb8wIdzELtrGY8COia6lVQSvCnzANieW2BPbTkTrClEnS5vAfTxUVSnCYPcg70hnWzNoqGtHHWXaIkG905GSIglA4SBU5Yf-EQlqpNd-oRyCNLtoSfPQSl_Fu483cTGbYphf1HGkouVJWgh8xfLOYb0YTHi0QUdhVWjo5xswOsPDqL2oz46Nx_J1UvGCb2uulGCOdlcJ6isSQvPUVxV7eutdPXAkXIEDR8gKjI0HrxH3FC7xUauKnLlUKiHx4rSKMmQhScbAirMkWcihE2g3YOcip9aseMiiAO-SYwYo6rxL-6W8dSn2lV6tdS0NbN-3sn7wQY62yz0412rv-hG0nr0faSdw4pv7N5mARMvq6VHLf-ftAKUp0i-F3AA8tUcqPKk77gFNlJInn7dzYX1bXpOwYw0lZyIBbzKwioOtP1DESFl8kk9W2OKVql0DIoHaPsvSF5L9Ysd0N4lUJXG8KcIf74OES5NvldqR7zhULIGgcEE391d5w3G9J2A5zlrdg_otM69F6-UlLcItV3VE3_zHIQNmZiJOF-pErpBdt7RUnW9OMDl_A8tVa-d_t_HPBG4Uf1e3__91e61ZiMA2DVugZrCysT1HZojSTz6nkSAI9ZSt7DTDUxJuwTuWXRuYHH_39Gd6rUDHFQM0-qZ_E1-HK1P13mgKNQo', 'SETUP_TIME': 180, 'SSH_AUTHORIZED_KEYS': ['ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0DXCqWbugQT5CIGKflvFT3RFocubrdVeZoH9iCMZOuYvUC1LfOxmmdO5OKqFKoKFb3ShnHJgBo9VzCYDst6nLUeZsIpDk0bEZ0O3rUOD59zIgXpkFyLQr3Su9sCWOOjto0RRtCjcvgwRsjDOVUEZ8GSCQ0aZMLvaLEELDjEmh57MUW5CkWGlNSy/X97zq4vSPBN268vCkixAZrq3J8oIrAAJBLro7WPGtlf8bbxrtjaO9Ma1ZElAosFn+HKgDAiLENFDa52ohPZAgJwrYCtZHqNBu5ll4+wCIMcimjoEojMXvuZ6bw4BkFn6UYXX/ONmhEPc3bLAhwm6MExnV1n8Z fedora@preserve-appmigration-workmachine'], 'VOLUME_SIZE': '8G', 'WORKER_IMAGE': 'brew.registry.redhat.io/rh-osbs/quay-quay-builder-rhel8', 'WORKER_TAG': 'v3.8.11'} builder stdout | 2023-08-08 05:58:30,926 [55] [DEBUG] [buildman.manager.executor] Kubernetes request: GET https://api.whu412bm02.qe.devcluster.openshift.com:6443/apis/batch/v1/namespaces/builder/jobs: {'params': {'labelSelector': 'build,time,manager,quay-sha'}, 'verify': '/conf/stack/extra_ca_certs/build_cluster.crt', 'timeout': 20, 'headers': {'Authorization': 'Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IkkwS1RXc3dNZ3ZLOFlZUE1pWFMwSDRGUFNZRkx3Tk5abEEtRlNiLTJsWlkifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjIl0sImV4cCI6MTY5MTU1MTE0OSwiaWF0IjoxNjkxNDY0NzQ5LCJpc3MiOiJodHRwczovL2t1YmVybmV0ZXMuZGVmYXVsdC5zdmMiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImJ1aWxkZXIiLCJzZXJ2aWNlYWNjb3VudCI6eyJuYW1lIjoicXVheS1idWlsZGVyIiwidWlkIjoiOThkMGRmNGYtMGRiMS00M2VkLWExMmYtY2Q4MDVjZDRiMzg3In19LCJuYmYiOjE2OTE0NjQ3NDksInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpidWlsZGVyOnF1YXktYnVpbGRlciJ9.LpmO7SvKJb8wIdzELtrGY8COia6lVQSvCnzANieW2BPbTkTrClEnS5vAfTxUVSnCYPcg70hnWzNoqGtHHWXaIkG905GSIglA4SBU5Yf-EQlqpNd-oRyCNLtoSfPQSl_Fu483cTGbYphf1HGkouVJWgh8xfLOYb0YTHi0QUdhVWjo5xswOsPDqL2oz46Nx_J1UvGCb2uulGCOdlcJ6isSQvPUVxV7eutdPXAkXIEDR8gKjI0HrxH3FC7xUauKnLlUKiHx4rSKMmQhScbAirMkWcihE2g3YOcip9aseMiiAO-SYwYo6rxL-6W8dSn2lV6tdS0NbN-3sn7wQY62yz0412rv-hG0nr0faSdw4pv7N5mARMvq6VHLf-ftAKUp0i-F3AA8tUcqPKk77gFNlJInn7dzYX1bXpOwYw0lZyIBbzKwioOtP1DESFl8kk9W2OKVql0DIoHaPsvSF5L9Ysd0N4lUJXG8KcIf74OES5NvldqR7zhULIGgcEE391d5w3G9J2A5zlrdg_otM69F6-UlLcItV3VE3_zHIQNmZiJOF-pErpBdt7RUnW9OMDl_A8tVa-d_t_HPBG4Uf1e3__91e61ZiMA2DVugZrCysT1HZojSTz6nkSAI9ZSt7DTDUxJuwTuWXRuYHH_39Gd6rUDHFQM0-qZ_E1-HK1P13mgKNQo'}} builder stdout | 2023-08-08 05:58:30,929 [55] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.whu412bm02.qe.devcluster.openshift.com:6443 builder stdout | 2023-08-08 05:58:30,942 [55] [DEBUG] [urllib3.connectionpool] https://api.whu412bm02.qe.devcluster.openshift.com:6443 "GET /apis/batch/v1/namespaces/builder/jobs?labelSelector=build%2Ctime%2Cmanager%2Cquay-sha HTTP/1.1" 200 93 builder stdout | 2023-08-08 05:58:30,942 [55] [DEBUG] [buildman.manager.executor] Kubernetes response: 200: {"kind":"JobList","apiVersion":"batch/v1","metadata":{"resourceVersion":"94920"},"items":[]} builder stdout | 2023-08-08 05:58:30,956 [55] [DEBUG] [data.database] Disconnecting from database. builder stdout | 2023-08-08 05:58:30,957 [55] [DEBUG] [buildman.manager.ephemeral] Starting builder for job buildman/production/building/d5b40e12-f92c-42e5-9c25-9e76946bc5a3 with selected executor: KubernetesExecutor builder stdout | 2023-08-08 05:58:30,966 [55] [DEBUG] [buildman.manager.executor] Using Kubernetes Distribution: openshift builder stdout | 2023-08-08 05:58:30,966 [55] [DEBUG] [buildman.manager.executor] Generated kubernetes resource: builder stdout | {'apiVersion': 'batch/v1', 'kind': 'Job', 'metadata': {'namespace': 'builder', 'generateName': 'd5b40e12-f92c-42e5-9c25-9e76946bc5a3-', 'labels': {'build': 'd5b40e12-f92c-42e5-9c25-9e76946bc5a3', 'time': '2023-08-08-05', 'manager': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'quay-sha': 'none'}}, 'spec': {'activeDeadlineSeconds': 7200, 'backoffLimit': 1, 'template': {'metadata': {'labels': {'build': 'd5b40e12-f92c-42e5-9c25-9e76946bc5a3', 'time': '2023-08-08-05', 'manager': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'quay-sha': 'none'}}, 'spec': {'imagePullSecrets': [{'name': 'builder'}], 'restartPolicy': 'Never', 'dnsPolicy': 'Default', 'containers': [{'name': 'builder', 'imagePullPolicy': 'Always', 'image': 'brew.registry.redhat.io/rh-osbs/quay-quay-builder-qemu-rhcos-rhel8:v3.8.11', 'securityContext': {'privileged': True}, 'env': [{'name': 'USERDATA', 'value': '{"ignition": {"version": "3.0.0"}, "passwd": {"users": [{"sshAuthorizedKeys": ["ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0DXCqWbugQT5CIGKflvFT3RFocubrdVeZoH9iCMZOuYvUC1LfOxmmdO5OKqFKoKFb3ShnHJgBo9VzCYDst6nLUeZsIpDk0bEZ0O3rUOD59zIgXpkFyLQr3Su9sCWOOjto0RRtCjcvgwRsjDOVUEZ8GSCQ0aZMLvaLEELDjEmh57MUW5CkWGlNSy/X97zq4vSPBN268vCkixAZrq3J8oIrAAJBLro7WPGtlf8bbxrtjaO9Ma1ZElAosFn+HKgDAiLENFDa52ohPZAgJwrYCtZHqNBu5ll4+wCIMcimjoEojMXvuZ6bw4BkFn6UYXX/ONmhEPc3bLAhwm6MExnV1n8Z fedora@preserve-appmigration-workmachine"], "groups": ["sudo"], "name": "core"}]}, "storage": {"files": [{"path": "/etc/pki/ca-trust-source/anchors/ssl.cert", "contents": {"source": "data:,-----BEGIN%20CERTIFICATE-----%0AMIIEvzCCA6egAwIBAgIUc9wns0k8nRREVi6bQcvd2wRCzO4wDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNDA3MjkwMzIyMTBaMIGWMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMT0wOwYDVQQDDDR3aHVxdWF5%0ALmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tMIIB%0AIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsy5HsEhnVYz/DZ391UsZzXQq%0Au%2BhjZXtmuiLNFBo8Es6r4taub1McaWAvG4VaN2%2BYvU4mdditXqUSbFTBvI6iZKjv%0ADsWxlvnAsdZnwgzt0bRQZppmjnQ1QutMeUusGISOwS%2BLlF2TSSO6wK/HImIQ1Jh7%0Afh9oTF0zx/4bFLQta1Ky%2Bl8pVM6WLtzitNPvZPWVTW9tfNvmJlJewofD6c9GKyWx%0A4uQf4Sxj0/4YCaRx6FMt9hEvv3DS3utR8AwXeZIJrTBpeAz0vTkxqIU8VrV3LFqh%0AAcHDXZ7huxJc9U4ftiGSDDbffUgLC6xfmThMmMHNgQ8Hhp05Csb589rvd83rvwID%0AAQABo4IBCDCCAQQwCQYDVR0TBAIwADALBgNVHQ8EBAMCBeAwgekGA1UdEQSB4TCB%0A3oJWcXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBw%0Acy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb22CTnF1YXly%0AZWdpc3RyeS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLndodTQxM29zcDAzLnFl%0ALmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbYI0d2h1cXVheS5hcHBzLndodTQxM29z%0AcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbTANBgkqhkiG9w0BAQsFAAOC%0AAQEAscZyf1npNNXMtfoMdANDmCdW8XcKehwANN0IWrfdtwxACZGsi5oIoQoEz8f2%0AF0%2BJNo8mXSpcMzcZhTdwMfJD7Hght2fTfMu5HLhQox9Q9MRCgxJjBDuX/IYCbe7N%0AZwJhYaiLOZSAC7l86MWYYnpIQycW6zTeWJeFbOHk6GVnZXVaP7cuYx79024ci8fU%0A8RW66KmKTKMV3h6SUYXxrwYI3XLzkMAcxirT1HsQRNQ1xFkof5ME/WiBoJOT0Drd%0AudlYggLbKbwAN1NIceNVHhpdcDlbuZcR3TO94uW3eesuUTExi46Zy/4Z2nuWkMBX%0AdH3jQVjLowU2tFfaqoVcs/aRBQ%3D%3D%0A-----END%20CERTIFICATE-----%0A-----BEGIN%20CERTIFICATE-----%0AMIIEATCCAumgAwIBAgIURnWvVoz3LSuldzMbVPP3l3u8AQMwDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNjA1MjgwMzIyMTBaMIGPMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMTYwNAYDVQQDDC0uYXBwcy53%0AaHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb20wggEiMA0GCSqG%0ASIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBkCJbJZLIkpkILzDbOU0wFxZTvPZxEHU%2B%0AtYufbSnBKx0oP0aKY47ErxF3v5B3uDB9yRrTauMz4Iss%2BU6pfx%2B0VFTVMDyRUs8k%0ArVkFuBgbpvWZhde5aUe1SxLjtxCvQFOPjGZCO/FRPWGnR26JkkyIBR9x8AphfMYo%0ARmWK2ZMJCeD7S3I9KObQLodnzFVGK%2BfVWFb/yUmDwEzLNk8fqMyxVq7RBfWoLWcN%0AYvGW6UXV25%2BMz3Vm56N/pXezymkwL56AgIXbcajbT9ZyZhxXt4X2oojYat6dizID%0Adddjo4eJMbXDo8X29TO96ITcbvIHW9fvPMSpyGnaQfo07U%2BpnBpNAgMBAAGjUzBR%0AMB0GA1UdDgQWBBR6voxj/OVNOUSzhcXPLci9ntZ9CTAfBgNVHSMEGDAWgBR6voxj%0A/OVNOUSzhcXPLci9ntZ9CTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA%0AA4IBAQAStdlvFjRWynmKODpzPoglw21weUiskBjbP%2Bfh%2BQIDxKYn1UQOEvtdlD3L%0AXiiy6vd85IZj5dZlugw/kiILfL/CIciPojO3rg/rWHG/IYf6J5NsII%2BfQkN%2BT/4W%0A8usnu3ByxwjUsguvWJ1hgEWDfh%2BV69XmvIXlr3cXz7i6lLag3DHff6wzHFctfuFt%0AAk14d71LRqByLmfb8cSbBUHMTmaDw/ISYojxUhDkxa4%2BhNUDQ%2BonTF0B7oF%2BSyhO%0Au%2B/XMWf6H9fjbjbSTh29vO5wfSNn9uQJIGUs4wcQBhqPTsloOR9RC3kthZOpyWcO%0A6IAol2GDPZg1P1Uxt3oUHaJ9fW/m%0A-----END%20CERTIFICATE-----%0A"}, "mode": 420}, {"path": "/etc/containers/certs.d/quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/client.crt", "contents": {"source": "data:,-----BEGIN%20CERTIFICATE-----%0AMIIEvzCCA6egAwIBAgIUc9wns0k8nRREVi6bQcvd2wRCzO4wDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNDA3MjkwMzIyMTBaMIGWMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMT0wOwYDVQQDDDR3aHVxdWF5%0ALmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tMIIB%0AIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsy5HsEhnVYz/DZ391UsZzXQq%0Au%2BhjZXtmuiLNFBo8Es6r4taub1McaWAvG4VaN2%2BYvU4mdditXqUSbFTBvI6iZKjv%0ADsWxlvnAsdZnwgzt0bRQZppmjnQ1QutMeUusGISOwS%2BLlF2TSSO6wK/HImIQ1Jh7%0Afh9oTF0zx/4bFLQta1Ky%2Bl8pVM6WLtzitNPvZPWVTW9tfNvmJlJewofD6c9GKyWx%0A4uQf4Sxj0/4YCaRx6FMt9hEvv3DS3utR8AwXeZIJrTBpeAz0vTkxqIU8VrV3LFqh%0AAcHDXZ7huxJc9U4ftiGSDDbffUgLC6xfmThMmMHNgQ8Hhp05Csb589rvd83rvwID%0AAQABo4IBCDCCAQQwCQYDVR0TBAIwADALBgNVHQ8EBAMCBeAwgekGA1UdEQSB4TCB%0A3oJWcXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBw%0Acy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb22CTnF1YXly%0AZWdpc3RyeS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLndodTQxM29zcDAzLnFl%0ALmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbYI0d2h1cXVheS5hcHBzLndodTQxM29z%0AcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbTANBgkqhkiG9w0BAQsFAAOC%0AAQEAscZyf1npNNXMtfoMdANDmCdW8XcKehwANN0IWrfdtwxACZGsi5oIoQoEz8f2%0AF0%2BJNo8mXSpcMzcZhTdwMfJD7Hght2fTfMu5HLhQox9Q9MRCgxJjBDuX/IYCbe7N%0AZwJhYaiLOZSAC7l86MWYYnpIQycW6zTeWJeFbOHk6GVnZXVaP7cuYx79024ci8fU%0A8RW66KmKTKMV3h6SUYXxrwYI3XLzkMAcxirT1HsQRNQ1xFkof5ME/WiBoJOT0Drd%0AudlYggLbKbwAN1NIceNVHhpdcDlbuZcR3TO94uW3eesuUTExi46Zy/4Z2nuWkMBX%0AdH3jQVjLowU2tFfaqoVcs/aRBQ%3D%3D%0A-----END%20CERTIFICATE-----%0A-----BEGIN%20CERTIFICATE-----%0AMIIEATCCAumgAwIBAgIURnWvVoz3LSuldzMbVPP3l3u8AQMwDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNjA1MjgwMzIyMTBaMIGPMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMTYwNAYDVQQDDC0uYXBwcy53%0AaHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb20wggEiMA0GCSqG%0ASIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBkCJbJZLIkpkILzDbOU0wFxZTvPZxEHU%2B%0AtYufbSnBKx0oP0aKY47ErxF3v5B3uDB9yRrTauMz4Iss%2BU6pfx%2B0VFTVMDyRUs8k%0ArVkFuBgbpvWZhde5aUe1SxLjtxCvQFOPjGZCO/FRPWGnR26JkkyIBR9x8AphfMYo%0ARmWK2ZMJCeD7S3I9KObQLodnzFVGK%2BfVWFb/yUmDwEzLNk8fqMyxVq7RBfWoLWcN%0AYvGW6UXV25%2BMz3Vm56N/pXezymkwL56AgIXbcajbT9ZyZhxXt4X2oojYat6dizID%0Adddjo4eJMbXDo8X29TO96ITcbvIHW9fvPMSpyGnaQfo07U%2BpnBpNAgMBAAGjUzBR%0AMB0GA1UdDgQWBBR6voxj/OVNOUSzhcXPLci9ntZ9CTAfBgNVHSMEGDAWgBR6voxj%0A/OVNOUSzhcXPLci9ntZ9CTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA%0AA4IBAQAStdlvFjRWynmKODpzPoglw21weUiskBjbP%2Bfh%2BQIDxKYn1UQOEvtdlD3L%0AXiiy6vd85IZj5dZlugw/kiILfL/CIciPojO3rg/rWHG/IYf6J5NsII%2BfQkN%2BT/4W%0A8usnu3ByxwjUsguvWJ1hgEWDfh%2BV69XmvIXlr3cXz7i6lLag3DHff6wzHFctfuFt%0AAk14d71LRqByLmfb8cSbBUHMTmaDw/ISYojxUhDkxa4%2BhNUDQ%2BonTF0B7oF%2BSyhO%0Au%2B/XMWf6H9fjbjbSTh29vO5wfSNn9uQJIGUs4wcQBhqPTsloOR9RC3kthZOpyWcO%0A6IAol2GDPZg1P1Uxt3oUHaJ9fW/m%0A-----END%20CERTIFICATE-----%0A"}, "mode": 420}, {"overwrite": true, "path": "/etc/containers/registries.conf", "contents": {"source": "data:,unqualified-search-registries%20%3D%20%5B%22registry.fedoraproject.org%22%2C%20%22registry.access.redhat.com%22%2C%20%22docker.io%22%2C%20%22quay.io%22%5D%0Ashort-name-mode%3D%22permissive%22"}, "mode": 444}, {"path": "/etc/hostname", "contents": {"source": "data:,d5b40e12-f92c-42e5-9c25-9e76946bc5a3"}, "mode": 420}, {"overwrite": true, "path": "/etc/zincati/config.d/90-disable-auto-updates.toml", "contents": {"source": "data:,%5Bupdates%5D%0Aenabled%20%3D%20false"}, "mode": 420}, {"path": "/usr/local/bin/disable-aws-metadata.sh", "contents": {"source": "data:,%23%21/bin/bash%0Aiptables%20-t%20nat%20-I%20PREROUTING%20-p%20tcp%20-d%20169.254.169.254%20--dport%2080%20-j%20DNAT%20--to-destination%201.1.1.1"}, "mode": 493}, {"path": "/root/overrides.list", "contents": {"source": "data:,TOKEN%3DeyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBwcy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb206NDQzIiwibmJmIjoxNjkxNDc0MzEwLCJpYXQiOjE2OTE0NzQzMTAsImV4cCI6MTY5MTQ3NDk0MCwic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOnt9LCJjb250ZXh0Ijp7InRva2VuX3R5cGUiOiJidWlsZF9qb2JfcmVnaXN0cmF0aW9uIiwiYnVpbGRfaWQiOiJkNWI0MGUxMi1mOTJjLTQyZTUtOWMyNS05ZTc2OTQ2YmM1YTMiLCJqb2JfaWQiOiJidWlsZG1hbi9wcm9kdWN0aW9uL2J1aWxkaW5nL2Q1YjQwZTEyLWY5MmMtNDJlNS05YzI1LTllNzY5NDZiYzVhMyIsImV4cGlyYXRpb24iOjYzMH19.edeygMDPTthtMjWlvQZxW-M9gMCefJaE0Excnt2S2X2rsLit7AZY8p8T2e7dXHsqyEtL1ypciGx35EESYXuuJk1xZNbOyhgNEl2T7HvG-5Sq0NGeLPhzisCSn8Ql3K6mL4-myieNJzNIUgBxSg9qgYiC_4fdpg9B-45Tvt7lgYC0CMO7v1RYZSct6coaJr4CYmFrSpncxVN-tpuaq0Dgb_XqnvaQoJO3WXGMlw2baeVqLptDq3q52escGHTLmJKjxs-apvDqcdhh7xkiTXbQuKIrzIpACv-iDrku-lALX44p9LyvuQvoI2k-kl_G0RoebXCNpesYVoNeC-qfhgzdlA%0ASERVER%3Dquayregistry-quay-builder-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com%3A443%0A%0A%23%20TODO%3A%20Remove%20this%20eventually%0AGODEBUG%3Dx509ignoreCN%3D0"}, "mode": 420}]}, "systemd": {"units": [{"name": "quay-builder.service", "enabled": true, "contents": "[Unit]\\nWants=podman.service network-online.target\\nAfter=podman.service network-online.target\\nRequires=podman.service\\n\\n[Service]\\nType=oneshot\\nTimeoutStartSec=10800\\nTimeoutStopSec=2000\\n\\nExecStartPre=/usr/bin/podman login -u |76db245a-0d75-11e7-a544-28d244ea5a6d.j9is.751956 -p eyJhbGciOiJSUzUxMiJ9.eyJzdWIiOiI3YTE0OWIwNWZkZTI0ZWMwYWQ3ZTIwMjQwZmMyYmU2MCJ9.W-4araN4Cus25Nl39ZOp5MmTZBzeJQlGVhaxh1bRWX2hcAhCaoflBf4l2U71ewD16F9My20HPSSpqIJJ1zydnrXKA731WRknwHYetVZ929_vVNUtHcbCvDYTzCseg781DihDbiWm2pK6mbUKLUEXJXzVaAo7hTbXQsgs63EpKGF3bJyN0M76ZnU8BNkMkaj_cWN0ir8As1vf_OEJudIiSmxxUruF-VMANbTkE50UlP-oh2VCqypBxQ81R5VLu4YsdXfNUwl5jdl14T9MGK0OTw7WAt5VdHBQqr3lsjd4WDWJqigemyrdYM5BC1XZXh8j8v_CfU-yaKXNugTtr8LYk1n6Wso1ThxL9eDUfYFvLRYLoJYizGfOjPcoPJugRIh-tS6ki7zy33-Ya4pkSLLX-OdN4Vq1UJvfE13X8PAOSAbWKF7-7dk1DG2LNW1M2UtEelmyVg6KQQ7cRlmMXp7QouaKO9vDKmudDH_2H9ZVrsXhSFTNmoS_YVkDbxqTs6ugtx5TLxbKYYdNnUygDVKprNdRZSZc-adH4JPuRBroA6CTup13PyW92EtePRN6G2IpXInMC9kkApur5qUynddIscb5KnCaYuKNeqQBCQDI7YxP6CCAfm1obf1nUWbDc2uk4DSNitRZJ84hsLBCLTABTyR0aCGYszCeU4zQ-rn5cc8 brew.registry.redhat.io\\nExecStart=/usr/bin/podman run --user 0 --rm --privileged --env-file /root/overrides.list -v /var/run/podman/podman.sock:/var/run/podman/podman.sock -v /etc/pki/ca-trust-source/anchors:/certs -e CONTAINER_RUNTIME=podman -e DOCKER_HOST=unix:/var/run/podman/podman.sock --name quay-builder brew.registry.redhat.io/rh-osbs/quay-quay-builder-rhel8:v3.8.11\\n\\n[Install]\\nWantedBy=multi-user.target"}, {"name": "systemd-journal-gatewayd.socket", "enabled": true, "contents": "[Unit]\\nDescription=Journal Gateway Service Socket\\n[Socket]\\nListenStream=/var/run/journald.sock\\nService=systemd-journal-gatewayd.service\\n[Install]\\nWantedBy=sockets.target"}, {"name": "disable-aws-metadata.service", "enabled": true, "contents": "[Unit]\\nDescription=Disable AWS metadata service\\nBefore=network-pre.target\\nWants=network-pre.target\\n[Service]\\nType=oneshot\\nExecStart=/usr/local/bin/disable-aws-metadata.sh\\nRemainAfterExit=yes\\n[Install]\\nWantedBy=multi-user.target"}, {"name": "machine-lifetime.service", "enabled": true, "contents": "[Unit]\\nDescription=Machine Lifetime Service\\n[Service]\\nType=oneshot\\nExecStart=/bin/sh -xc \\"/bin/sleep 10800; /usr/bin/systemctl --no-block poweroff\\"\\n[Install]\\nWantedBy=multi-user.target"}]}}'}, {'name': 'VM_MEMORY', 'value': '4G'}, {'name': 'VM_VOLUME_SIZE', 'value': '8G'}], 'resources': {'requests': {'memory': '3968Mi', 'cpu': '500m'}, 'limits': {'memory': '5120Mi', 'cpu': '1000m'}}}], 'automountServiceAccountToken': False, 'serviceAccount': 'quay-builder', 'enableServiceLinks': False, 'nodeSelector': {'type': 'builder'}}}}} builder stdout | 2023-08-08 05:58:30,967 [55] [DEBUG] [buildman.manager.executor] Using service account token for Kubernetes authentication builder stdout | 2023-08-08 05:58:30,967 [55] [DEBUG] [buildman.manager.executor] Executor config: {'BUILDER_NAMESPACE': 'builder', 'BUILDER_VM_CONTAINER_IMAGE': 'brew.registry.redhat.io/rh-osbs/quay-quay-builder-qemu-rhcos-rhel8:v3.8.11', 'CONTAINER_CPU_LIMITS': '1000m', 'CONTAINER_CPU_REQUEST': '500m', 'CONTAINER_MEMORY_LIMITS': '5120Mi', 'CONTAINER_MEMORY_REQUEST': '3968Mi', 'CONTAINER_RUNTIME': 'podman', 'DEBUG': True, 'EXECUTOR': 'kubernetes', 'K8S_API_SERVER': 'api.whu412bm02.qe.devcluster.openshift.com:6443', 'K8S_API_TLS_CA': '/conf/stack/extra_ca_certs/build_cluster.crt', 'KUBERNETES_DISTRIBUTION': 'openshift', 'MINIMUM_RETRY_THRESHOLD': 0, 'NODE_SELECTOR_LABEL_KEY': 'type', 'NODE_SELECTOR_LABEL_VALUE': 'builder', 'QUAY_PASSWORD': 'eyJhbGciOiJSUzUxMiJ9.eyJzdWIiOiI3YTE0OWIwNWZkZTI0ZWMwYWQ3ZTIwMjQwZmMyYmU2MCJ9.W-4araN4Cus25Nl39ZOp5MmTZBzeJQlGVhaxh1bRWX2hcAhCaoflBf4l2U71ewD16F9My20HPSSpqIJJ1zydnrXKA731WRknwHYetVZ929_vVNUtHcbCvDYTzCseg781DihDbiWm2pK6mbUKLUEXJXzVaAo7hTbXQsgs63EpKGF3bJyN0M76ZnU8BNkMkaj_cWN0ir8As1vf_OEJudIiSmxxUruF-VMANbTkE50UlP-oh2VCqypBxQ81R5VLu4YsdXfNUwl5jdl14T9MGK0OTw7WAt5VdHBQqr3lsjd4WDWJqigemyrdYM5BC1XZXh8j8v_CfU-yaKXNugTtr8LYk1n6Wso1ThxL9eDUfYFvLRYLoJYizGfOjPcoPJugRIh-tS6ki7zy33-Ya4pkSLLX-OdN4Vq1UJvfE13X8PAOSAbWKF7-7dk1DG2LNW1M2UtEelmyVg6KQQ7cRlmMXp7QouaKO9vDKmudDH_2H9ZVrsXhSFTNmoS_YVkDbxqTs6ugtx5TLxbKYYdNnUygDVKprNdRZSZc-adH4JPuRBroA6CTup13PyW92EtePRN6G2IpXInMC9kkApur5qUynddIscb5KnCaYuKNeqQBCQDI7YxP6CCAfm1obf1nUWbDc2uk4DSNitRZJ84hsLBCLTABTyR0aCGYszCeU4zQ-rn5cc8', 'QUAY_USERNAME': '|76db245a-0d75-11e7-a544-28d244ea5a6d.j9is.751956', 'SERVICE_ACCOUNT_NAME': 'quay-builder', 'SERVICE_ACCOUNT_TOKEN': 'eyJhbGciOiJSUzI1NiIsImtpZCI6IkkwS1RXc3dNZ3ZLOFlZUE1pWFMwSDRGUFNZRkx3Tk5abEEtRlNiLTJsWlkifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjIl0sImV4cCI6MTY5MTU1MTE0OSwiaWF0IjoxNjkxNDY0NzQ5LCJpc3MiOiJodHRwczovL2t1YmVybmV0ZXMuZGVmYXVsdC5zdmMiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImJ1aWxkZXIiLCJzZXJ2aWNlYWNjb3VudCI6eyJuYW1lIjoicXVheS1idWlsZGVyIiwidWlkIjoiOThkMGRmNGYtMGRiMS00M2VkLWExMmYtY2Q4MDVjZDRiMzg3In19LCJuYmYiOjE2OTE0NjQ3NDksInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpidWlsZGVyOnF1YXktYnVpbGRlciJ9.LpmO7SvKJb8wIdzELtrGY8COia6lVQSvCnzANieW2BPbTkTrClEnS5vAfTxUVSnCYPcg70hnWzNoqGtHHWXaIkG905GSIglA4SBU5Yf-EQlqpNd-oRyCNLtoSfPQSl_Fu483cTGbYphf1HGkouVJWgh8xfLOYb0YTHi0QUdhVWjo5xswOsPDqL2oz46Nx_J1UvGCb2uulGCOdlcJ6isSQvPUVxV7eutdPXAkXIEDR8gKjI0HrxH3FC7xUauKnLlUKiHx4rSKMmQhScbAirMkWcihE2g3YOcip9aseMiiAO-SYwYo6rxL-6W8dSn2lV6tdS0NbN-3sn7wQY62yz0412rv-hG0nr0faSdw4pv7N5mARMvq6VHLf-ftAKUp0i-F3AA8tUcqPKk77gFNlJInn7dzYX1bXpOwYw0lZyIBbzKwioOtP1DESFl8kk9W2OKVql0DIoHaPsvSF5L9Ysd0N4lUJXG8KcIf74OES5NvldqR7zhULIGgcEE391d5w3G9J2A5zlrdg_otM69F6-UlLcItV3VE3_zHIQNmZiJOF-pErpBdt7RUnW9OMDl_A8tVa-d_t_HPBG4Uf1e3__91e61ZiMA2DVugZrCysT1HZojSTz6nkSAI9ZSt7DTDUxJuwTuWXRuYHH_39Gd6rUDHFQM0-qZ_E1-HK1P13mgKNQo', 'SETUP_TIME': 180, 'SSH_AUTHORIZED_KEYS': ['ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0DXCqWbugQT5CIGKflvFT3RFocubrdVeZoH9iCMZOuYvUC1LfOxmmdO5OKqFKoKFb3ShnHJgBo9VzCYDst6nLUeZsIpDk0bEZ0O3rUOD59zIgXpkFyLQr3Su9sCWOOjto0RRtCjcvgwRsjDOVUEZ8GSCQ0aZMLvaLEELDjEmh57MUW5CkWGlNSy/X97zq4vSPBN268vCkixAZrq3J8oIrAAJBLro7WPGtlf8bbxrtjaO9Ma1ZElAosFn+HKgDAiLENFDa52ohPZAgJwrYCtZHqNBu5ll4+wCIMcimjoEojMXvuZ6bw4BkFn6UYXX/ONmhEPc3bLAhwm6MExnV1n8Z fedora@preserve-appmigration-workmachine'], 'VOLUME_SIZE': '8G', 'WORKER_IMAGE': 'brew.registry.redhat.io/rh-osbs/quay-quay-builder-rhel8', 'WORKER_TAG': 'v3.8.11'} builder stdout | 2023-08-08 05:58:30,967 [55] [DEBUG] [buildman.manager.executor] Kubernetes request: POST https://api.whu412bm02.qe.devcluster.openshift.com:6443/apis/batch/v1/namespaces/builder/jobs: {'json': {'apiVersion': 'batch/v1', 'kind': 'Job', 'metadata': {'namespace': 'builder', 'generateName': 'd5b40e12-f92c-42e5-9c25-9e76946bc5a3-', 'labels': {'build': 'd5b40e12-f92c-42e5-9c25-9e76946bc5a3', 'time': '2023-08-08-05', 'manager': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'quay-sha': 'none'}}, 'spec': {'activeDeadlineSeconds': 7200, 'backoffLimit': 1, 'template': {'metadata': {'labels': {'build': 'd5b40e12-f92c-42e5-9c25-9e76946bc5a3', 'time': '2023-08-08-05', 'manager': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'quay-sha': 'none'}}, 'spec': {'imagePullSecrets': [{'name': 'builder'}], 'restartPolicy': 'Never', 'dnsPolicy': 'Default', 'containers': [{'name': 'builder', 'imagePullPolicy': 'Always', 'image': 'brew.registry.redhat.io/rh-osbs/quay-quay-builder-qemu-rhcos-rhel8:v3.8.11', 'securityContext': {'privileged': True}, 'env': [{'name': 'USERDATA', 'value': '{"ignition": {"version": "3.0.0"}, "passwd": {"users": [{"sshAuthorizedKeys": ["ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0DXCqWbugQT5CIGKflvFT3RFocubrdVeZoH9iCMZOuYvUC1LfOxmmdO5OKqFKoKFb3ShnHJgBo9VzCYDst6nLUeZsIpDk0bEZ0O3rUOD59zIgXpkFyLQr3Su9sCWOOjto0RRtCjcvgwRsjDOVUEZ8GSCQ0aZMLvaLEELDjEmh57MUW5CkWGlNSy/X97zq4vSPBN268vCkixAZrq3J8oIrAAJBLro7WPGtlf8bbxrtjaO9Ma1ZElAosFn+HKgDAiLENFDa52ohPZAgJwrYCtZHqNBu5ll4+wCIMcimjoEojMXvuZ6bw4BkFn6UYXX/ONmhEPc3bLAhwm6MExnV1n8Z fedora@preserve-appmigration-workmachine"], "groups": ["sudo"], "name": "core"}]}, "storage": {"files": [{"path": "/etc/pki/ca-trust-source/anchors/ssl.cert", "contents": {"source": "data:,-----BEGIN%20CERTIFICATE-----%0AMIIEvzCCA6egAwIBAgIUc9wns0k8nRREVi6bQcvd2wRCzO4wDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNDA3MjkwMzIyMTBaMIGWMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMT0wOwYDVQQDDDR3aHVxdWF5%0ALmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tMIIB%0AIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsy5HsEhnVYz/DZ391UsZzXQq%0Au%2BhjZXtmuiLNFBo8Es6r4taub1McaWAvG4VaN2%2BYvU4mdditXqUSbFTBvI6iZKjv%0ADsWxlvnAsdZnwgzt0bRQZppmjnQ1QutMeUusGISOwS%2BLlF2TSSO6wK/HImIQ1Jh7%0Afh9oTF0zx/4bFLQta1Ky%2Bl8pVM6WLtzitNPvZPWVTW9tfNvmJlJewofD6c9GKyWx%0A4uQf4Sxj0/4YCaRx6FMt9hEvv3DS3utR8AwXeZIJrTBpeAz0vTkxqIU8VrV3LFqh%0AAcHDXZ7huxJc9U4ftiGSDDbffUgLC6xfmThMmMHNgQ8Hhp05Csb589rvd83rvwID%0AAQABo4IBCDCCAQQwCQYDVR0TBAIwADALBgNVHQ8EBAMCBeAwgekGA1UdEQSB4TCB%0A3oJWcXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBw%0Acy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb22CTnF1YXly%0AZWdpc3RyeS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLndodTQxM29zcDAzLnFl%0ALmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbYI0d2h1cXVheS5hcHBzLndodTQxM29z%0AcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbTANBgkqhkiG9w0BAQsFAAOC%0AAQEAscZyf1npNNXMtfoMdANDmCdW8XcKehwANN0IWrfdtwxACZGsi5oIoQoEz8f2%0AF0%2BJNo8mXSpcMzcZhTdwMfJD7Hght2fTfMu5HLhQox9Q9MRCgxJjBDuX/IYCbe7N%0AZwJhYaiLOZSAC7l86MWYYnpIQycW6zTeWJeFbOHk6GVnZXVaP7cuYx79024ci8fU%0A8RW66KmKTKMV3h6SUYXxrwYI3XLzkMAcxirT1HsQRNQ1xFkof5ME/WiBoJOT0Drd%0AudlYggLbKbwAN1NIceNVHhpdcDlbuZcR3TO94uW3eesuUTExi46Zy/4Z2nuWkMBX%0AdH3jQVjLowU2tFfaqoVcs/aRBQ%3D%3D%0A-----END%20CERTIFICATE-----%0A-----BEGIN%20CERTIFICATE-----%0AMIIEATCCAumgAwIBAgIURnWvVoz3LSuldzMbVPP3l3u8AQMwDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNjA1MjgwMzIyMTBaMIGPMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMTYwNAYDVQQDDC0uYXBwcy53%0AaHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb20wggEiMA0GCSqG%0ASIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBkCJbJZLIkpkILzDbOU0wFxZTvPZxEHU%2B%0AtYufbSnBKx0oP0aKY47ErxF3v5B3uDB9yRrTauMz4Iss%2BU6pfx%2B0VFTVMDyRUs8k%0ArVkFuBgbpvWZhde5aUe1SxLjtxCvQFOPjGZCO/FRPWGnR26JkkyIBR9x8AphfMYo%0ARmWK2ZMJCeD7S3I9KObQLodnzFVGK%2BfVWFb/yUmDwEzLNk8fqMyxVq7RBfWoLWcN%0AYvGW6UXV25%2BMz3Vm56N/pXezymkwL56AgIXbcajbT9ZyZhxXt4X2oojYat6dizID%0Adddjo4eJMbXDo8X29TO96ITcbvIHW9fvPMSpyGnaQfo07U%2BpnBpNAgMBAAGjUzBR%0AMB0GA1UdDgQWBBR6voxj/OVNOUSzhcXPLci9ntZ9CTAfBgNVHSMEGDAWgBR6voxj%0A/OVNOUSzhcXPLci9ntZ9CTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA%0AA4IBAQAStdlvFjRWynmKODpzPoglw21weUiskBjbP%2Bfh%2BQIDxKYn1UQOEvtdlD3L%0AXiiy6vd85IZj5dZlugw/kiILfL/CIciPojO3rg/rWHG/IYf6J5NsII%2BfQkN%2BT/4W%0A8usnu3ByxwjUsguvWJ1hgEWDfh%2BV69XmvIXlr3cXz7i6lLag3DHff6wzHFctfuFt%0AAk14d71LRqByLmfb8cSbBUHMTmaDw/ISYojxUhDkxa4%2BhNUDQ%2BonTF0B7oF%2BSyhO%0Au%2B/XMWf6H9fjbjbSTh29vO5wfSNn9uQJIGUs4wcQBhqPTsloOR9RC3kthZOpyWcO%0A6IAol2GDPZg1P1Uxt3oUHaJ9fW/m%0A-----END%20CERTIFICATE-----%0A"}, "mode": 420}, {"path": "/etc/containers/certs.d/quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/client.crt", "contents": {"source": "data:,-----BEGIN%20CERTIFICATE-----%0AMIIEvzCCA6egAwIBAgIUc9wns0k8nRREVi6bQcvd2wRCzO4wDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNDA3MjkwMzIyMTBaMIGWMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMT0wOwYDVQQDDDR3aHVxdWF5%0ALmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tMIIB%0AIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsy5HsEhnVYz/DZ391UsZzXQq%0Au%2BhjZXtmuiLNFBo8Es6r4taub1McaWAvG4VaN2%2BYvU4mdditXqUSbFTBvI6iZKjv%0ADsWxlvnAsdZnwgzt0bRQZppmjnQ1QutMeUusGISOwS%2BLlF2TSSO6wK/HImIQ1Jh7%0Afh9oTF0zx/4bFLQta1Ky%2Bl8pVM6WLtzitNPvZPWVTW9tfNvmJlJewofD6c9GKyWx%0A4uQf4Sxj0/4YCaRx6FMt9hEvv3DS3utR8AwXeZIJrTBpeAz0vTkxqIU8VrV3LFqh%0AAcHDXZ7huxJc9U4ftiGSDDbffUgLC6xfmThMmMHNgQ8Hhp05Csb589rvd83rvwID%0AAQABo4IBCDCCAQQwCQYDVR0TBAIwADALBgNVHQ8EBAMCBeAwgekGA1UdEQSB4TCB%0A3oJWcXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBw%0Acy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb22CTnF1YXly%0AZWdpc3RyeS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLndodTQxM29zcDAzLnFl%0ALmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbYI0d2h1cXVheS5hcHBzLndodTQxM29z%0AcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbTANBgkqhkiG9w0BAQsFAAOC%0AAQEAscZyf1npNNXMtfoMdANDmCdW8XcKehwANN0IWrfdtwxACZGsi5oIoQoEz8f2%0AF0%2BJNo8mXSpcMzcZhTdwMfJD7Hght2fTfMu5HLhQox9Q9MRCgxJjBDuX/IYCbe7N%0AZwJhYaiLOZSAC7l86MWYYnpIQycW6zTeWJeFbOHk6GVnZXVaP7cuYx79024ci8fU%0A8RW66KmKTKMV3h6SUYXxrwYI3XLzkMAcxirT1HsQRNQ1xFkof5ME/WiBoJOT0Drd%0AudlYggLbKbwAN1NIceNVHhpdcDlbuZcR3TO94uW3eesuUTExi46Zy/4Z2nuWkMBX%0AdH3jQVjLowU2tFfaqoVcs/aRBQ%3D%3D%0A-----END%20CERTIFICATE-----%0A-----BEGIN%20CERTIFICATE-----%0AMIIEATCCAumgAwIBAgIURnWvVoz3LSuldzMbVPP3l3u8AQMwDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNjA1MjgwMzIyMTBaMIGPMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMTYwNAYDVQQDDC0uYXBwcy53%0AaHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb20wggEiMA0GCSqG%0ASIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBkCJbJZLIkpkILzDbOU0wFxZTvPZxEHU%2B%0AtYufbSnBKx0oP0aKY47ErxF3v5B3uDB9yRrTauMz4Iss%2BU6pfx%2B0VFTVMDyRUs8k%0ArVkFuBgbpvWZhde5aUe1SxLjtxCvQFOPjGZCO/FRPWGnR26JkkyIBR9x8AphfMYo%0ARmWK2ZMJCeD7S3I9KObQLodnzFVGK%2BfVWFb/yUmDwEzLNk8fqMyxVq7RBfWoLWcN%0AYvGW6UXV25%2BMz3Vm56N/pXezymkwL56AgIXbcajbT9ZyZhxXt4X2oojYat6dizID%0Adddjo4eJMbXDo8X29TO96ITcbvIHW9fvPMSpyGnaQfo07U%2BpnBpNAgMBAAGjUzBR%0AMB0GA1UdDgQWBBR6voxj/OVNOUSzhcXPLci9ntZ9CTAfBgNVHSMEGDAWgBR6voxj%0A/OVNOUSzhcXPLci9ntZ9CTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA%0AA4IBAQAStdlvFjRWynmKODpzPoglw21weUiskBjbP%2Bfh%2BQIDxKYn1UQOEvtdlD3L%0AXiiy6vd85IZj5dZlugw/kiILfL/CIciPojO3rg/rWHG/IYf6J5NsII%2BfQkN%2BT/4W%0A8usnu3ByxwjUsguvWJ1hgEWDfh%2BV69XmvIXlr3cXz7i6lLag3DHff6wzHFctfuFt%0AAk14d71LRqByLmfb8cSbBUHMTmaDw/ISYojxUhDkxa4%2BhNUDQ%2BonTF0B7oF%2BSyhO%0Au%2B/XMWf6H9fjbjbSTh29vO5wfSNn9uQJIGUs4wcQBhqPTsloOR9RC3kthZOpyWcO%0A6IAol2GDPZg1P1Uxt3oUHaJ9fW/m%0A-----END%20CERTIFICATE-----%0A"}, "mode": 420}, {"overwrite": true, "path": "/etc/containers/registries.conf", "contents": {"source": "data:,unqualified-search-registries%20%3D%20%5B%22registry.fedoraproject.org%22%2C%20%22registry.access.redhat.com%22%2C%20%22docker.io%22%2C%20%22quay.io%22%5D%0Ashort-name-mode%3D%22permissive%22"}, "mode": 444}, {"path": "/etc/hostname", "contents": {"source": "data:,d5b40e12-f92c-42e5-9c25-9e76946bc5a3"}, "mode": 420}, {"overwrite": true, "path": "/etc/zincati/config.d/90-disable-auto-updates.toml", "contents": {"source": "data:,%5Bupdates%5D%0Aenabled%20%3D%20false"}, "mode": 420}, {"path": "/usr/local/bin/disable-aws-metadata.sh", "contents": {"source": "data:,%23%21/bin/bash%0Aiptables%20-t%20nat%20-I%20PREROUTING%20-p%20tcp%20-d%20169.254.169.254%20--dport%2080%20-j%20DNAT%20--to-destination%201.1.1.1"}, "mode": 493}, {"path": "/root/overrides.list", "contents": {"source": "data:,TOKEN%3DeyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBwcy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb206NDQzIiwibmJmIjoxNjkxNDc0MzEwLCJpYXQiOjE2OTE0NzQzMTAsImV4cCI6MTY5MTQ3NDk0MCwic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOnt9LCJjb250ZXh0Ijp7InRva2VuX3R5cGUiOiJidWlsZF9qb2JfcmVnaXN0cmF0aW9uIiwiYnVpbGRfaWQiOiJkNWI0MGUxMi1mOTJjLTQyZTUtOWMyNS05ZTc2OTQ2YmM1YTMiLCJqb2JfaWQiOiJidWlsZG1hbi9wcm9kdWN0aW9uL2J1aWxkaW5nL2Q1YjQwZTEyLWY5MmMtNDJlNS05YzI1LTllNzY5NDZiYzVhMyIsImV4cGlyYXRpb24iOjYzMH19.edeygMDPTthtMjWlvQZxW-M9gMCefJaE0Excnt2S2X2rsLit7AZY8p8T2e7dXHsqyEtL1ypciGx35EESYXuuJk1xZNbOyhgNEl2T7HvG-5Sq0NGeLPhzisCSn8Ql3K6mL4-myieNJzNIUgBxSg9qgYiC_4fdpg9B-45Tvt7lgYC0CMO7v1RYZSct6coaJr4CYmFrSpncxVN-tpuaq0Dgb_XqnvaQoJO3WXGMlw2baeVqLptDq3q52escGHTLmJKjxs-apvDqcdhh7xkiTXbQuKIrzIpACv-iDrku-lALX44p9LyvuQvoI2k-kl_G0RoebXCNpesYVoNeC-qfhgzdlA%0ASERVER%3Dquayregistry-quay-builder-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com%3A443%0A%0A%23%20TODO%3A%20Remove%20this%20eventually%0AGODEBUG%3Dx509ignoreCN%3D0"}, "mode": 420}]}, "systemd": {"units": [{"name": "quay-builder.service", "enabled": true, "contents": "[Unit]\\nWants=podman.service network-online.target\\nAfter=podman.service network-online.target\\nRequires=podman.service\\n\\n[Service]\\nType=oneshot\\nTimeoutStartSec=10800\\nTimeoutStopSec=2000\\n\\nExecStartPre=/usr/bin/podman login -u |76db245a-0d75-11e7-a544-28d244ea5a6d.j9is.751956 -p eyJhbGciOiJSUzUxMiJ9.eyJzdWIiOiI3YTE0OWIwNWZkZTI0ZWMwYWQ3ZTIwMjQwZmMyYmU2MCJ9.W-4araN4Cus25Nl39ZOp5MmTZBzeJQlGVhaxh1bRWX2hcAhCaoflBf4l2U71ewD16F9My20HPSSpqIJJ1zydnrXKA731WRknwHYetVZ929_vVNUtHcbCvDYTzCseg781DihDbiWm2pK6mbUKLUEXJXzVaAo7hTbXQsgs63EpKGF3bJyN0M76ZnU8BNkMkaj_cWN0ir8As1vf_OEJudIiSmxxUruF-VMANbTkE50UlP-oh2VCqypBxQ81R5VLu4YsdXfNUwl5jdl14T9MGK0OTw7WAt5VdHBQqr3lsjd4WDWJqigemyrdYM5BC1XZXh8j8v_CfU-yaKXNugTtr8LYk1n6Wso1ThxL9eDUfYFvLRYLoJYizGfOjPcoPJugRIh-tS6ki7zy33-Ya4pkSLLX-OdN4Vq1UJvfE13X8PAOSAbWKF7-7dk1DG2LNW1M2UtEelmyVg6KQQ7cRlmMXp7QouaKO9vDKmudDH_2H9ZVrsXhSFTNmoS_YVkDbxqTs6ugtx5TLxbKYYdNnUygDVKprNdRZSZc-adH4JPuRBroA6CTup13PyW92EtePRN6G2IpXInMC9kkApur5qUynddIscb5KnCaYuKNeqQBCQDI7YxP6CCAfm1obf1nUWbDc2uk4DSNitRZJ84hsLBCLTABTyR0aCGYszCeU4zQ-rn5cc8 brew.registry.redhat.io\\nExecStart=/usr/bin/podman run --user 0 --rm --privileged --env-file /root/overrides.list -v /var/run/podman/podman.sock:/var/run/podman/podman.sock -v /etc/pki/ca-trust-source/anchors:/certs -e CONTAINER_RUNTIME=podman -e DOCKER_HOST=unix:/var/run/podman/podman.sock --name quay-builder brew.registry.redhat.io/rh-osbs/quay-quay-builder-rhel8:v3.8.11\\n\\n[Install]\\nWantedBy=multi-user.target"}, {"name": "systemd-journal-gatewayd.socket", "enabled": true, "contents": "[Unit]\\nDescription=Journal Gateway Service Socket\\n[Socket]\\nListenStream=/var/run/journald.sock\\nService=systemd-journal-gatewayd.service\\n[Install]\\nWantedBy=sockets.target"}, {"name": "disable-aws-metadata.service", "enabled": true, "contents": "[Unit]\\nDescription=Disable AWS metadata service\\nBefore=network-pre.target\\nWants=network-pre.target\\n[Service]\\nType=oneshot\\nExecStart=/usr/local/bin/disable-aws-metadata.sh\\nRemainAfterExit=yes\\n[Install]\\nWantedBy=multi-user.target"}, {"name": "machine-lifetime.service", "enabled": true, "contents": "[Unit]\\nDescription=Machine Lifetime Service\\n[Service]\\nType=oneshot\\nExecStart=/bin/sh -xc \\"/bin/sleep 10800; /usr/bin/systemctl --no-block poweroff\\"\\n[Install]\\nWantedBy=multi-user.target"}]}}'}, {'name': 'VM_MEMORY', 'value': '4G'}, {'name': 'VM_VOLUME_SIZE', 'value': '8G'}], 'resources': {'requests': {'memory': '3968Mi', 'cpu': '500m'}, 'limits': {'memory': '5120Mi', 'cpu': '1000m'}}}], 'automountServiceAccountToken': False, 'serviceAccount': 'quay-builder', 'enableServiceLinks': False, 'nodeSelector': {'type': 'builder'}}}}}, 'verify': '/conf/stack/extra_ca_certs/build_cluster.crt', 'timeout': 20, 'headers': {'Authorization': 'Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IkkwS1RXc3dNZ3ZLOFlZUE1pWFMwSDRGUFNZRkx3Tk5abEEtRlNiLTJsWlkifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjIl0sImV4cCI6MTY5MTU1MTE0OSwiaWF0IjoxNjkxNDY0NzQ5LCJpc3MiOiJodHRwczovL2t1YmVybmV0ZXMuZGVmYXVsdC5zdmMiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImJ1aWxkZXIiLCJzZXJ2aWNlYWNjb3VudCI6eyJuYW1lIjoicXVheS1idWlsZGVyIiwidWlkIjoiOThkMGRmNGYtMGRiMS00M2VkLWExMmYtY2Q4MDVjZDRiMzg3In19LCJuYmYiOjE2OTE0NjQ3NDksInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpidWlsZGVyOnF1YXktYnVpbGRlciJ9.LpmO7SvKJb8wIdzELtrGY8COia6lVQSvCnzANieW2BPbTkTrClEnS5vAfTxUVSnCYPcg70hnWzNoqGtHHWXaIkG905GSIglA4SBU5Yf-EQlqpNd-oRyCNLtoSfPQSl_Fu483cTGbYphf1HGkouVJWgh8xfLOYb0YTHi0QUdhVWjo5xswOsPDqL2oz46Nx_J1UvGCb2uulGCOdlcJ6isSQvPUVxV7eutdPXAkXIEDR8gKjI0HrxH3FC7xUauKnLlUKiHx4rSKMmQhScbAirMkWcihE2g3YOcip9aseMiiAO-SYwYo6rxL-6W8dSn2lV6tdS0NbN-3sn7wQY62yz0412rv-hG0nr0faSdw4pv7N5mARMvq6VHLf-ftAKUp0i-F3AA8tUcqPKk77gFNlJInn7dzYX1bXpOwYw0lZyIBbzKwioOtP1DESFl8kk9W2OKVql0DIoHaPsvSF5L9Ysd0N4lUJXG8KcIf74OES5NvldqR7zhULIGgcEE391d5w3G9J2A5zlrdg_otM69F6-UlLcItV3VE3_zHIQNmZiJOF-pErpBdt7RUnW9OMDl_A8tVa-d_t_HPBG4Uf1e3__91e61ZiMA2DVugZrCysT1HZojSTz6nkSAI9ZSt7DTDUxJuwTuWXRuYHH_39Gd6rUDHFQM0-qZ_E1-HK1P13mgKNQo'}} builder stdout | 2023-08-08 05:58:30,968 [55] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.whu412bm02.qe.devcluster.openshift.com:6443 builder stdout | 2023-08-08 05:58:30,983 [55] [DEBUG] [urllib3.connectionpool] https://api.whu412bm02.qe.devcluster.openshift.com:6443 "POST /apis/batch/v1/namespaces/builder/jobs HTTP/1.1" 201 None builder stdout | 2023-08-08 05:58:30,984 [55] [DEBUG] [buildman.manager.executor] Kubernetes response: 201: {"kind":"Job","apiVersion":"batch/v1","metadata":{"name":"d5b40e12-f92c-42e5-9c25-9e76946bc5a3-8rlkw","generateName":"d5b40e12-f92c-42e5-9c25-9e76946bc5a3-","namespace":"builder","uid":"19b3e962-e942-411c-b1dd-391a0b90ec46","resourceVersion":"94921","generation":1,"creationTimestamp":"2023-08-08T05:58:30Z","labels":{"build":"d5b40e12-f92c-42e5-9c25-9e76946bc5a3","manager":"quayregistry-quay-app-758d8cbbb5-6xw9b","quay-sha":"none","time":"2023-08-08-05"},"annotations":{"batch.kubernetes.io/job-tracking":""},"managedFields":[{"manager":"python-requests","operation":"Update","apiVersion":"batch/v1","time":"2023-08-08T05:58:30Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:build":{},"f:manager":{},"f:quay-sha":{},"f:time":{}}},"f:spec":{"f:activeDeadlineSeconds":{},"f:backoffLimit":{},"f:completionMode":{},"f:completions":{},"f:parallelism":{},"f:suspend":{},"f:template":{"f:metadata":{"f:labels":{".":{},"f:build":{},"f:manager":{},"f:quay-sha":{},"f:time":{}}},"f:spec":{"f:automountServiceAccountToken":{},"f:containers":{"k:{\"name\":\"builder\"}":{".":{},"f:env":{".":{},"k:{\"name\":\"USERDATA\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"VM_MEMORY\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"VM_VOLUME_SIZE\"}":{".":{},"f:name":{},"f:value":{}}},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{".":{},"f:limits":{".":{},"f:cpu":{},"f:memory":{}},"f:requests":{".":{},"f:cpu":{},"f:memory":{}}},"f:securityContext":{".":{},"f:privileged":{}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:imagePullSecrets":{".":{},"k:{\"name\":\"builder\"}":{}},"f:nodeSelector":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:serviceAccount":{},"f:serviceAccountName":{},"f:terminationGracePeriodSeconds":{}}}}}}]},"spec":{"parallelism":1,"completions":1,"activeDeadlineSeconds":7200,"backoffLimit":1,"selector":{"matchLabels":{"controller-uid":"19b3e962-e942-411c-b1dd-391a0b90ec46"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"build":"d5b40e12-f92c-42e5-9c25-9e76946bc5a3","controller-uid":"19b3e962-e942-411c-b1dd-391a0b90ec46","job-name":"d5b40e12-f92c-42e5-9c25-9e76946bc5a3-8rlkw","manager":"quayregistry-quay-app-758d8cbbb5-6xw9b","quay-sha":"none","time":"2023-08-08-05"}},"spec":{"containers":[{"name":"builder","image":"brew.registry.redhat.io/rh-osbs/quay-quay-builder-qemu-rhcos-rhel8:v3.8.11","env":[{"name":"USERDATA","value":"{\"ignition\": {\"version\": \"3.0.0\"}, \"passwd\": {\"users\": [{\"sshAuthorizedKeys\": [\"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0DXCqWbugQT5CIGKflvFT3RFocubrdVeZoH9iCMZOuYvUC1LfOxmmdO5OKqFKoKFb3ShnHJgBo9VzCYDst6nLUeZsIpDk0bEZ0O3rUOD59zIgXpkFyLQr3Su9sCWOOjto0RRtCjcvgwRsjDOVUEZ8GSCQ0aZMLvaLEELDjEmh57MUW5CkWGlNSy/X97zq4vSPBN268vCkixAZrq3J8oIrAAJBLro7WPGtlf8bbxrtjaO9Ma1ZElAosFn+HKgDAiLENFDa52ohPZAgJwrYCtZHqNBu5ll4+wCIMcimjoEojMXvuZ6bw4BkFn6UYXX/ONmhEPc3bLAhwm6MExnV1n8Z fedora@preserve-appmigration-workmachine\"], \"groups\": [\"sudo\"], \"name\": \"core\"}]}, \"storage\": {\"files\": [{\"path\": \"/etc/pki/ca-trust-source/anchors/ssl.cert\", \"contents\": {\"source\": \"data:,-----BEGIN%20CERTIFICATE-----%0AMIIEvzCCA6egAwIBAgIUc9wns0k8nRREVi6bQcvd2wRCzO4wDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNDA3MjkwMzIyMTBaMIGWMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMT0wOwYDVQQDDDR3aHVxdWF5%0ALmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tMIIB%0AIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsy5HsEhnVYz/DZ391UsZzXQq%0Au%2BhjZXtmuiLNFBo8Es6r4taub1McaWAvG4VaN2%2BYvU4mdditXqUSbFTBvI6iZKjv%0ADsWxlvnAsdZnwgzt0bRQZppmjnQ1QutMeUusGISOwS%2BLlF2TSSO6wK/HImIQ1Jh7%0Afh9oTF0zx/4bFLQta1Ky%2Bl8pVM6WLtzitNPvZPWVTW9tfNvmJlJewofD6c9GKyWx%0A4uQf4Sxj0/4YCaRx6FMt9hEvv3DS3utR8AwXeZIJrTBpeAz0vTkxqIU8VrV3LFqh%0AAcHDXZ7huxJc9U4ftiGSDDbffUgLC6xfmThMmMHNgQ8Hhp05Csb589rvd83rvwID%0AAQABo4IBCDCCAQQwCQYDVR0TBAIwADALBgNVHQ8EBAMCBeAwgekGA1UdEQSB4TCB%0A3oJWcXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBw%0Acy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb22CTnF1YXly%0AZWdpc3RyeS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLndodTQxM29zcDAzLnFl%0ALmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbYI0d2h1cXVheS5hcHBzLndodTQxM29z%0AcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbTANBgkqhkiG9w0BAQsFAAOC%0AAQEAscZyf1npNNXMtfoMdANDmCdW8XcKehwANN0IWrfdtwxACZGsi5oIoQoEz8f2%0AF0%2BJNo8mXSpcMzcZhTdwMfJD7Hght2fTfMu5HLhQox9Q9MRCgxJjBDuX/IYCbe7N%0AZwJhYaiLOZSAC7l86MWYYnpIQycW6zTeWJeFbOHk6GVnZXVaP7cuYx79024ci8fU%0A8RW66KmKTKMV3h6SUYXxrwYI3XLzkMAcxirT1HsQRNQ1xFkof5ME/WiBoJOT0Drd%0AudlYggLbKbwAN1NIceNVHhpdcDlbuZcR3TO94uW3eesuUTExi46Zy/4Z2nuWkMBX%0AdH3jQVjLowU2tFfaqoVcs/aRBQ%3D%3D%0A-----END%20CERTIFICATE-----%0A-----BEGIN%20CERTIFICATE-----%0AMIIEATCCAumgAwIBAgIURnWvVoz3LSuldzMbVPP3l3u8AQMwDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNjA1MjgwMzIyMTBaMIGPMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMTYwNAYDVQQDDC0uYXBwcy53%0AaHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb20wggEiMA0GCSqG%0ASIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBkCJbJZLIkpkILzDbOU0wFxZTvPZxEHU%2B%0AtYufbSnBKx0oP0aKY47ErxF3v5B3uDB9yRrTauMz4Iss%2BU6pfx%2B0VFTVMDyRUs8k%0ArVkFuBgbpvWZhde5aUe1SxLjtxCvQFOPjGZCO/FRPWGnR26JkkyIBR9x8AphfMYo%0ARmWK2ZMJCeD7S3I9KObQLodnzFVGK%2BfVWFb/yUmDwEzLNk8fqMyxVq7RBfWoLWcN%0AYvGW6UXV25%2BMz3Vm56N/pXezymkwL56AgIXbcajbT9ZyZhxXt4X2oojYat6dizID%0Adddjo4eJMbXDo8X29TO96ITcbvIHW9fvPMSpyGnaQfo07U%2BpnBpNAgMBAAGjUzBR%0AMB0GA1UdDgQWBBR6voxj/OVNOUSzhcXPLci9ntZ9CTAfBgNVHSMEGDAWgBR6voxj%0A/OVNOUSzhcXPLci9ntZ9CTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA%0AA4IBAQAStdlvFjRWynmKODpzPoglw21weUiskBjbP%2Bfh%2BQIDxKYn1UQOEvtdlD3L%0AXiiy6vd85IZj5dZlugw/kiILfL/CIciPojO3rg/rWHG/IYf6J5NsII%2BfQkN%2BT/4W%0A8usnu3ByxwjUsguvWJ1hgEWDfh%2BV69XmvIXlr3cXz7i6lLag3DHff6wzHFctfuFt%0AAk14d71LRqByLmfb8cSbBUHMTmaDw/ISYojxUhDkxa4%2BhNUDQ%2BonTF0B7oF%2BSyhO%0Au%2B/XMWf6H9fjbjbSTh29vO5wfSNn9uQJIGUs4wcQBhqPTsloOR9RC3kthZOpyWcO%0A6IAol2GDPZg1P1Uxt3oUHaJ9fW/m%0A-----END%20CERTIFICATE-----%0A\"}, \"mode\": 420}, {\"path\": \"/etc/containers/certs.d/quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/client.crt\", \"contents\": {\"source\": \"data:,-----BEGIN%20CERTIFICATE-----%0AMIIEvzCCA6egAwIBAgIUc9wns0k8nRREVi6bQcvd2wRCzO4wDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNDA3MjkwMzIyMTBaMIGWMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMT0wOwYDVQQDDDR3aHVxdWF5%0ALmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tMIIB%0AIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsy5HsEhnVYz/DZ391UsZzXQq%0Au%2BhjZXtmuiLNFBo8Es6r4taub1McaWAvG4VaN2%2BYvU4mdditXqUSbFTBvI6iZKjv%0ADsWxlvnAsdZnwgzt0bRQZppmjnQ1QutMeUusGISOwS%2BLlF2TSSO6wK/HImIQ1Jh7%0Afh9oTF0zx/4bFLQta1Ky%2Bl8pVM6WLtzitNPvZPWVTW9tfNvmJlJewofD6c9GKyWx%0A4uQf4Sxj0/4YCaRx6FMt9hEvv3DS3utR8AwXeZIJrTBpeAz0vTkxqIU8VrV3LFqh%0AAcHDXZ7huxJc9U4ftiGSDDbffUgLC6xfmThMmMHNgQ8Hhp05Csb589rvd83rvwID%0AAQABo4IBCDCCAQQwCQYDVR0TBAIwADALBgNVHQ8EBAMCBeAwgekGA1UdEQSB4TCB%0A3oJWcXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBw%0Acy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb22CTnF1YXly%0AZWdpc3RyeS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLndodTQxM29zcDAzLnFl%0ALmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbYI0d2h1cXVheS5hcHBzLndodTQxM29z%0AcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0LmNvbTANBgkqhkiG9w0BAQsFAAOC%0AAQEAscZyf1npNNXMtfoMdANDmCdW8XcKehwANN0IWrfdtwxACZGsi5oIoQoEz8f2%0AF0%2BJNo8mXSpcMzcZhTdwMfJD7Hght2fTfMu5HLhQox9Q9MRCgxJjBDuX/IYCbe7N%0AZwJhYaiLOZSAC7l86MWYYnpIQycW6zTeWJeFbOHk6GVnZXVaP7cuYx79024ci8fU%0A8RW66KmKTKMV3h6SUYXxrwYI3XLzkMAcxirT1HsQRNQ1xFkof5ME/WiBoJOT0Drd%0AudlYggLbKbwAN1NIceNVHhpdcDlbuZcR3TO94uW3eesuUTExi46Zy/4Z2nuWkMBX%0AdH3jQVjLowU2tFfaqoVcs/aRBQ%3D%3D%0A-----END%20CERTIFICATE-----%0A-----BEGIN%20CERTIFICATE-----%0AMIIEATCCAumgAwIBAgIURnWvVoz3LSuldzMbVPP3l3u8AQMwDQYJKoZIhvcNAQEL%0ABQAwgY8xCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMQswCQYDVQQHDAJC%0ASjESMBAGA1UECgwJUXVheSB0ZWFtMRUwEwYDVQQLDAxRdWF5IFFFIFRlYW0xNjA0%0ABgNVBAMMLS5hcHBzLndodTQxM29zcDAzLnFlLmRldmNsdXN0ZXIub3BlbnNoaWZ0%0ALmNvbTAeFw0yMzA4MDgwMzIyMTBaFw0yNjA1MjgwMzIyMTBaMIGPMQswCQYDVQQG%0AEwJDTjEQMA4GA1UECAwHQmVpamluZzELMAkGA1UEBwwCQkoxEjAQBgNVBAoMCVF1%0AYXkgdGVhbTEVMBMGA1UECwwMUXVheSBRRSBUZWFtMTYwNAYDVQQDDC0uYXBwcy53%0AaHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb20wggEiMA0GCSqG%0ASIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBkCJbJZLIkpkILzDbOU0wFxZTvPZxEHU%2B%0AtYufbSnBKx0oP0aKY47ErxF3v5B3uDB9yRrTauMz4Iss%2BU6pfx%2B0VFTVMDyRUs8k%0ArVkFuBgbpvWZhde5aUe1SxLjtxCvQFOPjGZCO/FRPWGnR26JkkyIBR9x8AphfMYo%0ARmWK2ZMJCeD7S3I9KObQLodnzFVGK%2BfVWFb/yUmDwEzLNk8fqMyxVq7RBfWoLWcN%0AYvGW6UXV25%2BMz3Vm56N/pXezymkwL56AgIXbcajbT9ZyZhxXt4X2oojYat6dizID%0Adddjo4eJMbXDo8X29TO96ITcbvIHW9fvPMSpyGnaQfo07U%2BpnBpNAgMBAAGjUzBR%0AMB0GA1UdDgQWBBR6voxj/OVNOUSzhcXPLci9ntZ9CTAfBgNVHSMEGDAWgBR6voxj%0A/OVNOUSzhcXPLci9ntZ9CTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA%0AA4IBAQAStdlvFjRWynmKODpzPoglw21weUiskBjbP%2Bfh%2BQIDxKYn1UQOEvtdlD3L%0AXiiy6vd85IZj5dZlugw/kiILfL/CIciPojO3rg/rWHG/IYf6J5NsII%2BfQkN%2BT/4W%0A8usnu3ByxwjUsguvWJ1hgEWDfh%2BV69XmvIXlr3cXz7i6lLag3DHff6wzHFctfuFt%0AAk14d71LRqByLmfb8cSbBUHMTmaDw/ISYojxUhDkxa4%2BhNUDQ%2BonTF0B7oF%2BSyhO%0Au%2B/XMWf6H9fjbjbSTh29vO5wfSNn9uQJIGUs4wcQBhqPTsloOR9RC3kthZOpyWcO%0A6IAol2GDPZg1P1Uxt3oUHaJ9fW/m%0A-----END%20CERTIFICATE-----%0A\"}, \"mode\": 420}, {\"overwrite\": true, \"path\": \"/etc/containers/registries.conf\", \"contents\": {\"source\": \"data:,unqualified-search-registries%20%3D%20%5B%22registry.fedoraproject.org%22%2C%20%22registry.access.redhat.com%22%2C%20%22docker.io%22%2C%20%22quay.io%22%5D%0Ashort-name-mode%3D%22permissive%22\"}, \"mode\": 444}, {\"path\": \"/etc/hostname\", \"contents\": {\"source\": \"data:,d5b40e12-f92c-42e5-9c25-9e76946bc5a3\"}, \"mode\": 420}, {\"overwrite\": true, \"path\": \"/etc/zincati/config.d/90-disable-auto-updates.toml\", \"contents\": {\"source\": \"data:,%5Bupdates%5D%0Aenabled%20%3D%20false\"}, \"mode\": 420}, {\"path\": \"/usr/local/bin/disable-aws-metadata.sh\", \"contents\": {\"source\": \"data:,%23%21/bin/bash%0Aiptables%20-t%20nat%20-I%20PREROUTING%20-p%20tcp%20-d%20169.254.169.254%20--dport%2080%20-j%20DNAT%20--to-destination%201.1.1.1\"}, \"mode\": 493}, {\"path\": \"/root/overrides.list\", \"contents\": {\"source\": \"data:,TOKEN%3DeyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktYnVpbGRlci1xdWF5LWVudGVycHJpc2UuYXBwcy53aHU0MTNvc3AwMy5xZS5kZXZjbHVzdGVyLm9wZW5zaGlmdC5jb206NDQzIiwibmJmIjoxNjkxNDc0MzEwLCJpYXQiOjE2OTE0NzQzMTAsImV4cCI6MTY5MTQ3NDk0MCwic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOnt9LCJjb250ZXh0Ijp7InRva2VuX3R5cGUiOiJidWlsZF9qb2JfcmVnaXN0cmF0aW9uIiwiYnVpbGRfaWQiOiJkNWI0MGUxMi1mOTJjLTQyZTUtOWMyNS05ZTc2OTQ2YmM1YTMiLCJqb2JfaWQiOiJidWlsZG1hbi9wcm9kdWN0aW9uL2J1aWxkaW5nL2Q1YjQwZTEyLWY5MmMtNDJlNS05YzI1LTllNzY5NDZiYzVhMyIsImV4cGlyYXRpb24iOjYzMH19.edeygMDPTthtMjWlvQZxW-M9gMCefJaE0Excnt2S2X2rsLit7AZY8p8T2e7dXHsqyEtL1ypciGx35EESYXuuJk1xZNbOyhgNEl2T7HvG-5Sq0NGeLPhzisCSn8Ql3K6mL4-myieNJzNIUgBxSg9qgYiC_4fdpg9B-45Tvt7lgYC0CMO7v1RYZSct6coaJr4CYmFrSpncxVN-tpuaq0Dgb_XqnvaQoJO3WXGMlw2baeVqLptDq3q52escGHTLmJKjxs-apvDqcdhh7xkiTXbQuKIrzIpACv-iDrku-lALX44p9LyvuQvoI2k-kl_G0RoebXCNpesYVoNeC-qfhgzdlA%0ASERVER%3Dquayregistry-quay-builder-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com%3A443%0A%0A%23%20TODO%3A%20Remove%20this%20eventually%0AGODEBUG%3Dx509ignoreCN%3D0\"}, \"mode\": 420}]}, \"systemd\": {\"units\": [{\"name\": \"quay-builder.service\", \"enabled\": true, \"contents\": \"[Unit]\\nWants=podman.service network-online.target\\nAfter=podman.service network-online.target\\nRequires=podman.service\\n\\n[Service]\\nType=oneshot\\nTimeoutStartSec=10800\\nTimeoutStopSec=2000\\n\\nExecStartPre=/usr/bin/podman login -u |76db245a-0d75-11e7-a544-28d244ea5a6d.j9is.751956 -p eyJhbGciOiJSUzUxMiJ9.eyJzdWIiOiI3YTE0OWIwNWZkZTI0ZWMwYWQ3ZTIwMjQwZmMyYmU2MCJ9.W-4araN4Cus25Nl39ZOp5MmTZBzeJQlGVhaxh1bRWX2hcAhCaoflBf4l2U71ewD16F9My20HPSSpqIJJ1zydnrXKA731WRknwHYetVZ929_vVNUtHcbCvDYTzCseg781DihDbiWm2pK6mbUKLUEXJXzVaAo7hTbXQsgs63EpKGF3bJyN0M76ZnU8BNkMkaj_cWN0ir8As1vf_OEJudIiSmxxUruF-VMANbTkE50UlP-oh2VCqypBxQ81R5VLu4YsdXfNUwl5jdl14T9MGK0OTw7WAt5VdHBQqr3lsjd4WDWJqigemyrdYM5BC1XZXh8j8v_CfU-yaKXNugTtr8LYk1n6Wso1ThxL9eDUfYFvLRYLoJYizGfOjPcoPJugRIh-tS6ki7zy33-Ya4pkSLLX-OdN4Vq1UJvfE13X8PAOSAbWKF7-7dk1DG2LNW1M2UtEelmyVg6KQQ7cRlmMXp7QouaKO9vDKmudDH_2H9ZVrsXhSFTNmoS_YVkDbxqTs6ugtx5TLxbKYYdNnUygDVKprNdRZSZc-adH4JPuRBroA6CTup13PyW92EtePRN6G2IpXInMC9kkApur5qUynddIscb5KnCaYuKNeqQBCQDI7YxP6CCAfm1obf1nUWbDc2uk4DSNitRZJ84hsLBCLTABTyR0aCGYszCeU4zQ-rn5cc8 brew.registry.redhat.io\\nExecStart=/usr/bin/podman run --user 0 --rm --privileged --env-file /root/overrides.list -v /var/run/podman/podman.sock:/var/run/podman/podman.sock -v /etc/pki/ca-trust-source/anchors:/certs -e CONTAINER_RUNTIME=podman -e DOCKER_HOST=unix:/var/run/podman/podman.sock --name quay-builder brew.registry.redhat.io/rh-osbs/quay-quay-builder-rhel8:v3.8.11\\n\\n[Install]\\nWantedBy=multi-user.target\"}, {\"name\": \"systemd-journal-gatewayd.socket\", \"enabled\": true, \"contents\": \"[Unit]\\nDescription=Journal Gateway Service Socket\\n[Socket]\\nListenStream=/var/run/journald.sock\\nService=systemd-journal-gatewayd.service\\n[Install]\\nWantedBy=sockets.target\"}, {\"name\": \"disable-aws-metadata.service\", \"enabled\": true, \"contents\": \"[Unit]\\nDescription=Disable AWS metadata service\\nBefore=network-pre.target\\nWants=network-pre.target\\n[Service]\\nType=oneshot\\nExecStart=/usr/local/bin/disable-aws-metadata.sh\\nRemainAfterExit=yes\\n[Install]\\nWantedBy=multi-user.target\"}, {\"name\": \"machine-lifetime.service\", \"enabled\": true, \"contents\": \"[Unit]\\nDescription=Machine Lifetime Service\\n[Service]\\nType=oneshot\\nExecStart=/bin/sh -xc \\\"/bin/sleep 10800; /usr/bin/systemctl --no-block poweroff\\\"\\n[Install]\\nWantedBy=multi-user.target\"}]}}"},{"name":"VM_MEMORY","value":"4G"},{"name":"VM_VOLUME_SIZE","value":"8G"}],"resources":{"limits":{"cpu":"1","memory":"5Gi"},"requests":{"cpu":"500m","memory":"3968Mi"}},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always","securityContext":{"privileged":true}}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"Default","nodeSelector":{"type":"builder"},"serviceAccountName":"quay-builder","serviceAccount":"quay-builder","automountServiceAccountToken":false,"securityContext":{},"imagePullSecrets":[{"name":"builder"}],"schedulerName":"default-scheduler","enableServiceLinks":false}},"completionMode":"NonIndexed","suspend":false},"status":{}} builder stdout | 2023-08-08 05:58:30,985 [55] [DEBUG] [buildman.orchestrator] publishing event: {"event": 2, "key": "buildman/production/building/d5b40e12-f92c-42e5-9c25-9e76946bc5a3", "value": "{\"job_queue_item\": {\"id\": 2, \"body\": \"{\\\"build_uuid\\\": \\\"d5b40e12-f92c-42e5-9c25-9e76946bc5a3\\\", \\\"pull_credentials\\\": null}\", \"retries_remaining\": 2}, \"max_expiration\": 1691481510, \"last_heartbeat\": null, \"executor_name\": \"KubernetesExecutor\", \"execution_id\": \"d5b40e12-f92c-42e5-9c25-9e76946bc5a3-8rlkw\"}"} builder stdout | 2023-08-08 05:58:30,999 [55] [DEBUG] [data.database] Disconnecting from database. builder stdout | 2023-08-08 05:58:31,011 [55] [DEBUG] [data.database] Disconnecting from database. builder stdout | 2023-08-08 05:58:31,020 [55] [DEBUG] [buildman.manager.ephemeral] Job scheduled for job buildman/production/building/d5b40e12-f92c-42e5-9c25-9e76946bc5a3 with execution with ID d5b40e12-f92c-42e5-9c25-9e76946bc5a3-8rlkw on control plane KubernetesExecutor with max startup time of 180 builder stdout | 2023-08-08 05:58:31,021 [55] [DEBUG] [buildman.orchestrator] publishing event: {"event": 1, "key": "buildman/production/metric/d5b40e12-f92c-42e5-9c25-9e76946bc5a3", "value": "{\"executor_name\": \"KubernetesExecutor\", \"start_time\": 1691474310.9842985}"} builder stdout | 2023-08-08 05:58:31,021 [55] [DEBUG] [buildman.manager.ephemeral] Build job buildman/production/building/d5b40e12-f92c-42e5-9c25-9e76946bc5a3 scheduled. builder stdout | 2023-08-08 05:58:31,021 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:58:31,027 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:58:31,343 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:58:31,604 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:58:31,932 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:58:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:58:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:42.293757+00:00 (in 9.999416 seconds) chunkcleanupworker stdout | 2023-08-08 05:58:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:42 UTC)" (scheduled at 2023-08-08 05:58:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:58:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:58:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 32, 294843), True, datetime.datetime(2023, 8, 8, 5, 58, 32, 294843), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:58:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:58:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:58:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:58:33,041 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:58:33,044 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:58:33,048 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:58:33,051 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:58:33,054 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:58:33,057 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:58:33,063 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:58:33,067 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:58:33,070 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} exportactionlogsworker stdout | 2023-08-08 05:58:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:58:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:47.398551+00:00 (in 13.883942 seconds) exportactionlogsworker stdout | 2023-08-08 05:58:33,514 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:03:33 UTC)" (scheduled at 2023-08-08 05:58:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 05:58:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 58, 33, 515066), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:58:33,522 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 58, 33, 515066), True, datetime.datetime(2023, 8, 8, 5, 58, 33, 515066), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:58:33,524 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 58, 33, 515066), True, datetime.datetime(2023, 8, 8, 5, 58, 33, 515066), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 5, 58, 33, 515066), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 05:58:33,526 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:58:33,526 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:03:33 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:58:34,716 [198] [DEBUG] [app] Starting request: urn:request:25ffab22-cae7-491b-8517-b69f8cb14ce1 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:34,716 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:34,716 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:34,717 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:34,724 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:34,724 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:34,724 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:34,724 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:34,724 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:34,724 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:34,724 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:34,724 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:34,727 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:34,733 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:34,734 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:34,738 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 05:58:34,743 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:34,743 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:34,743 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:34,744 [198] [DEBUG] [app] Ending request: urn:request:25ffab22-cae7-491b-8517-b69f8cb14ce1 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:25ffab22-cae7-491b-8517-b69f8cb14ce1', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:34,745 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:34,745 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:34 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 2103 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:34 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/2.0" 200 2103 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 866 0.030) gunicorn-web stdout | 2023-08-08 05:58:35,136 [200] [DEBUG] [app] Starting request: urn:request:5b94ba3d-0bc6-4a45-9f33-bd416f275cc5 (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:35,136 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:35,136 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:35,137 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:35,144 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:35,144 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:35,144 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:35,144 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:35,144 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:35,144 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:35,144 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:35,144 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:35,146 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:35,152 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:35,154 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:35,157 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 10]) gunicorn-web stdout | 2023-08-08 05:58:35,163 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:35,163 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:35,163 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:35,164 [200] [DEBUG] [app] Ending request: urn:request:5b94ba3d-0bc6-4a45-9f33-bd416f275cc5 (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:5b94ba3d-0bc6-4a45-9f33-bd416f275cc5', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/?limit=10', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:35,164 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:35,165 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:35 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/1.0" 200 2103 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:35 +0000] "GET /api/v1/repository/build_test/github/build/?limit=10 HTTP/2.0" 200 2103 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 418 0.030) notificationworker stdout | 2023-08-08 05:58:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:58:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:45.371682+00:00 (in 9.999427 seconds) notificationworker stdout | 2023-08-08 05:58:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:45 UTC)" (scheduled at 2023-08-08 05:58:35.371682+00:00) notificationworker stdout | 2023-08-08 05:58:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:58:35,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 35, 372790), True, datetime.datetime(2023, 8, 8, 5, 58, 35, 372790), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:58:35,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:58:35,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:58:35,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:58:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:58:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:54.393627+00:00 (in 17.813295 seconds) namespacegcworker stdout | 2023-08-08 05:58:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:36 UTC)" (scheduled at 2023-08-08 05:58:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:58:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:58:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:58:37,179 [197] [DEBUG] [app] Starting request: urn:request:1c79e36f-a42f-4ead-91c8-608ab9548fe2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:58:37,180 [199] [DEBUG] [app] Starting request: urn:request:8696d09f-321f-4bce-ae6f-b4507bd3ebc8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:58:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:37,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:37,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:58:37,185 [207] [DEBUG] [app] Starting request: urn:request:0d371a6a-0f9c-4ad3-a856-0caa6487c943 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:58:37,185 [206] [DEBUG] [app] Starting request: urn:request:faf300b6-9afc-4fd3-a4f4-63a687e7ee20 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:58:37,185 [207] [DEBUG] [app] Ending request: urn:request:0d371a6a-0f9c-4ad3-a856-0caa6487c943 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0d371a6a-0f9c-4ad3-a856-0caa6487c943', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:58:37,186 [206] [DEBUG] [app] Ending request: urn:request:faf300b6-9afc-4fd3-a4f4-63a687e7ee20 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:faf300b6-9afc-4fd3-a4f4-63a687e7ee20', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:58:37,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:58:37,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:58:37,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:37,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:37,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:37,191 [199] [DEBUG] [app] Starting request: urn:request:4e8f0b27-ce3f-420c-bfac-5800b99ba4c1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:58:37,192 [199] [DEBUG] [app] Ending request: urn:request:4e8f0b27-ce3f-420c-bfac-5800b99ba4c1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4e8f0b27-ce3f-420c-bfac-5800b99ba4c1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:58:37,192 [200] [DEBUG] [app] Starting request: urn:request:95820565-ad62-4ca9-bc95-c0bf7e21b9ad (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:58:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:58:37,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:37,192 [200] [DEBUG] [app] Ending request: urn:request:95820565-ad62-4ca9-bc95-c0bf7e21b9ad (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:95820565-ad62-4ca9-bc95-c0bf7e21b9ad', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:58:37,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:58:37,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:58:37,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 05:58:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 05:58:37,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:58:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:07.197412+00:00 (in 29.999522 seconds) gunicorn-web stdout | 2023-08-08 05:58:37,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:58:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:07 UTC)" (scheduled at 2023-08-08 05:58:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:58:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 58, 37, 198838), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:37,203 [199] [DEBUG] [app] Ending request: urn:request:8696d09f-321f-4bce-ae6f-b4507bd3ebc8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8696d09f-321f-4bce-ae6f-b4507bd3ebc8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:58:37,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:37,204 [197] [DEBUG] [app] Ending request: urn:request:1c79e36f-a42f-4ead-91c8-608ab9548fe2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1c79e36f-a42f-4ead-91c8-608ab9548fe2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:58:37,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:37,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:58:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:58:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:58:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:58:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:58:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 05:58:37,206 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:58:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:58:37,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:07 UTC)" executed successfully gcworker stdout | 2023-08-08 05:58:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:58:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:49.417660+00:00 (in 9.999297 seconds) gcworker stdout | 2023-08-08 05:58:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:49 UTC)" (scheduled at 2023-08-08 05:58:39.417660+00:00) gcworker stdout | 2023-08-08 05:58:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264719419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:58:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:58:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:58:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:49 UTC)" executed successfully builder stdout | 2023-08-08 05:58:41,027 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:58:41,037 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:58:41,037 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:58:41,042 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:58:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:58:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:52.293757+00:00 (in 9.999480 seconds) chunkcleanupworker stdout | 2023-08-08 05:58:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:52 UTC)" (scheduled at 2023-08-08 05:58:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:58:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:58:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 42, 294848), True, datetime.datetime(2023, 8, 8, 5, 58, 42, 294848), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:58:42,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:58:42,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:58:42,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:52 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:58:43,618 [197] [DEBUG] [app] Starting request: urn:request:145f5a8e-7c66-4adb-87ae-9d84d4b35de3 (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:43,618 [200] [DEBUG] [app] Starting request: urn:request:68929fb9-e43b-4349-9fdd-00d21c78217c (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:43,619 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,619 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,619 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,619 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,619 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:43,619 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:43,627 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:43,627 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:43,627 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,627 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:43,627 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,627 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:43,627 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,627 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,627 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,627 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:43,627 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,628 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:43,628 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,628 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,628 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,628 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,629 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:43,630 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:43,636 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,636 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,636 [197] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:43,637 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:43,637 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:43,639 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [2, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:43,640 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [2, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 05:58:43,641 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:43,642 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,642 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,642 [197] [DEBUG] [app] Ending request: urn:request:145f5a8e-7c66-4adb-87ae-9d84d4b35de3 (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:145f5a8e-7c66-4adb-87ae-9d84d4b35de3', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github?includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:43,643 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:43 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/2.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 75 0.027) gunicorn-web stdout | 2023-08-08 05:58:43,644 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:43 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/1.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:58:43,645 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,645 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,645 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,646 [200] [DEBUG] [app] Ending request: urn:request:68929fb9-e43b-4349-9fdd-00d21c78217c (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3) {'endpoint': 'api.repositorybuildresource', 'request_id': 'urn:request:68929fb9-e43b-4349-9fdd-00d21c78217c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:43,647 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:43 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3 HTTP/2.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 908 0.030) gunicorn-web stdout | 2023-08-08 05:58:43,647 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:43 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3 HTTP/1.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:58:43,987 [197] [DEBUG] [app] Starting request: urn:request:bd6f9e6f-5dfd-4577-a674-86f99050a6e0 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:43,987 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,987 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,988 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:43,995 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:43,995 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:43,995 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,995 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:43,995 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:43,995 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,995 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,996 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:43,998 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:44,004 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:44,005 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:44,009 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:44,014 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:44,015 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:44,015 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:44,016 [197] [DEBUG] [app] Ending request: urn:request:bd6f9e6f-5dfd-4577-a674-86f99050a6e0 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'endpoint': 'api.repositorybuildstatus', 'request_id': 'urn:request:bd6f9e6f-5dfd-4577-a674-86f99050a6e0', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:44,016 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:44,017 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:44 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/1.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:44 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/2.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.032 441 0.032) gunicorn-web stdout | 2023-08-08 05:58:44,350 [197] [DEBUG] [app] Starting request: urn:request:68802fd3-cafd-4bb3-85a3-9722bcf48e32 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:44,350 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:44,350 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:44,351 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:44,357 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:44,357 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:44,357 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:44,358 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:44,358 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:44,358 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:44,358 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:44,358 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:44,360 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:44,367 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:44,368 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:44,370 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:44,371 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:44,377 [197] [DEBUG] [app] Ending request: urn:request:68802fd3-cafd-4bb3-85a3-9722bcf48e32 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'endpoint': 'api.repositorybuildlogs', 'request_id': 'urn:request:68802fd3-cafd-4bb3-85a3-9722bcf48e32', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=0', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs', 'parameters': {'start': '0'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:44,377 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:44,378 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:44 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=0 HTTP/1.0" 200 136 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:44 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=0 HTTP/2.0" 200 136 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 446 0.030) notificationworker stdout | 2023-08-08 05:58:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:58:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:49.178248+00:00 (in 3.806048 seconds) notificationworker stdout | 2023-08-08 05:58:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:55 UTC)" (scheduled at 2023-08-08 05:58:45.371682+00:00) notificationworker stdout | 2023-08-08 05:58:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:58:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 45, 372723), True, datetime.datetime(2023, 8, 8, 5, 58, 45, 372723), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:58:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:58:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:58:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:58:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:58:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:25.514341+00:00 (in 38.115259 seconds) exportactionlogsworker stdout | 2023-08-08 05:58:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:47 UTC)" (scheduled at 2023-08-08 05:58:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:58:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:58:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 47, 399562), True, datetime.datetime(2023, 8, 8, 5, 58, 47, 399562), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:58:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:58:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:58:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:58:47,409 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:58:48,647 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} notificationworker stdout | 2023-08-08 05:58:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:58:49,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:55.371682+00:00 (in 6.192848 seconds) notificationworker stdout | 2023-08-08 05:58:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:03:49 UTC)" (scheduled at 2023-08-08 05:58:49.178248+00:00) notificationworker stdout | 2023-08-08 05:58:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 58, 49, 179298), 'notification/%']) notificationworker stdout | 2023-08-08 05:58:49,188 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 58, 49, 179298), True, datetime.datetime(2023, 8, 8, 5, 58, 49, 179298), 0, 'notification/%']) notificationworker stdout | 2023-08-08 05:58:49,190 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 58, 49, 179298), True, datetime.datetime(2023, 8, 8, 5, 58, 49, 179298), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 5, 58, 49, 179298), 'notification/%']) notificationworker stdout | 2023-08-08 05:58:49,191 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:58:49,191 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:03:49 UTC)" executed successfully gcworker stdout | 2023-08-08 05:58:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:58:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:59.417660+00:00 (in 9.999375 seconds) gcworker stdout | 2023-08-08 05:58:49,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:59 UTC)" (scheduled at 2023-08-08 05:58:49.417660+00:00) gcworker stdout | 2023-08-08 05:58:49,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264729419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:58:49,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:58:49,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:58:49,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:58:59 UTC)" executed successfully securityworker stdout | 2023-08-08 05:58:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:58:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:58:56.890741+00:00 (in 7.202888 seconds) securityworker stdout | 2023-08-08 05:58:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:19 UTC)" (scheduled at 2023-08-08 05:58:49.687363+00:00) securityworker stdout | 2023-08-08 05:58:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:58:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:58:49,692 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:58:49,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:58:49,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:58:49,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:58:49,700 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:58:49,785 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 05:58:50,186 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 05:58:50,477 [197] [DEBUG] [app] Starting request: urn:request:4e753689-4010-4223-8ff3-37c6bf4b904d (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:50,477 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,477 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,478 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:50,484 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:50,485 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:50,485 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,485 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:50,485 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,485 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:50,485 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:50,485 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:50,487 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:50,493 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:50,494 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:50,498 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:50,502 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,503 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,503 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,503 [197] [DEBUG] [app] Ending request: urn:request:4e753689-4010-4223-8ff3-37c6bf4b904d (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'endpoint': 'api.repositorybuildstatus', 'request_id': 'urn:request:4e753689-4010-4223-8ff3-37c6bf4b904d', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:50,504 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:50 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/2.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 913 0.030) gunicorn-web stdout | 2023-08-08 05:58:50,505 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:50 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/1.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:58:50,816 [199] [DEBUG] [app] Starting request: urn:request:d4899880-725d-482b-9721-7e722cd79aa4 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:50,816 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,816 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,817 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:50,823 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:50,823 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:50,823 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,824 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:50,824 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,824 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:50,824 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:50,824 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:50,827 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:50,833 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:50,834 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:50,837 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:50,839 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:50,845 [199] [DEBUG] [app] Ending request: urn:request:d4899880-725d-482b-9721-7e722cd79aa4 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'endpoint': 'api.repositorybuildlogs', 'request_id': 'urn:request:d4899880-725d-482b-9721-7e722cd79aa4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs', 'parameters': {'start': '1'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:50,846 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:50 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/2.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.032 447 0.031) gunicorn-web stdout | 2023-08-08 05:58:50,846 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:50 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/1.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" builder stdout | 2023-08-08 05:58:51,051 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:58:51,061 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:58:51,061 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:58:51,066 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:52,179 [199] [DEBUG] [app] Starting request: urn:request:69889843-598e-460f-aba5-d89d5f4e9302 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:58:52,179 [198] [DEBUG] [app] Starting request: urn:request:7ed9e386-b385-40ee-b837-a743a8b6fc45 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:58:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:58:52,184 [203] [DEBUG] [app] Starting request: urn:request:51b7c559-b433-4756-832b-dd9949a52ab8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:58:52,185 [203] [DEBUG] [app] Ending request: urn:request:51b7c559-b433-4756-832b-dd9949a52ab8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:51b7c559-b433-4756-832b-dd9949a52ab8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:58:52,185 [205] [DEBUG] [app] Starting request: urn:request:01f19139-6877-486d-8293-a49907efe8fe (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:58:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:58:52,186 [205] [DEBUG] [app] Ending request: urn:request:01f19139-6877-486d-8293-a49907efe8fe (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:01f19139-6877-486d-8293-a49907efe8fe', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:58:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:58:52,186 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 05:58:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:52,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:52,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:58:52,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:52,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:58:52,190 [197] [DEBUG] [app] Starting request: urn:request:40c4cd5c-901a-4e32-b3b1-3497b59fb164 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:58:52,191 [197] [DEBUG] [app] Ending request: urn:request:40c4cd5c-901a-4e32-b3b1-3497b59fb164 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:40c4cd5c-901a-4e32-b3b1-3497b59fb164', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:58:52,191 [200] [DEBUG] [app] Starting request: urn:request:0c28d97e-17b5-4b1c-8c37-710ad01acb0f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:58:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:58:52,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:52,192 [200] [DEBUG] [app] Ending request: urn:request:0c28d97e-17b5-4b1c-8c37-710ad01acb0f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0c28d97e-17b5-4b1c-8c37-710ad01acb0f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:58:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:58:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:58:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:58:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2023-08-08 05:58:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:58:52,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:58:52,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:58:52,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:58:52,203 [199] [DEBUG] [app] Ending request: urn:request:69889843-598e-460f-aba5-d89d5f4e9302 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:69889843-598e-460f-aba5-d89d5f4e9302', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:58:52,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:52,203 [198] [DEBUG] [app] Ending request: urn:request:7ed9e386-b385-40ee-b837-a743a8b6fc45 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7ed9e386-b385-40ee-b837-a743a8b6fc45', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:58:52,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:52,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:58:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:58:52,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:58:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:58:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:58:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 05:58:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:58:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:02.293757+00:00 (in 9.999462 seconds) chunkcleanupworker stdout | 2023-08-08 05:58:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:02 UTC)" (scheduled at 2023-08-08 05:58:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:58:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:58:52,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 52, 294844), True, datetime.datetime(2023, 8, 8, 5, 58, 52, 294844), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:58:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:58:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:58:52,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:58:52,941 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:58:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:58:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:13.097375+00:00 (in 18.812772 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:58:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:54 UTC)" (scheduled at 2023-08-08 05:58:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:58:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:58:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:58:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:58:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:36.579841+00:00 (in 42.185673 seconds) namespacegcworker stdout | 2023-08-08 05:58:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:54 UTC)" (scheduled at 2023-08-08 05:58:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:58:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:58:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 54, 394771), True, datetime.datetime(2023, 8, 8, 5, 58, 54, 394771), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:58:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:58:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:58:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 05:59:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:58:54,779 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} gunicorn-secscan stdout | 2023-08-08 05:58:54,820 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:58:54,826 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:58:54,828 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} repositorygcworker stdout | 2023-08-08 05:58:54,888 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:58:55,158 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:58:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:58:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:02.178530+00:00 (in 6.806204 seconds) notificationworker stdout | 2023-08-08 05:58:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:05 UTC)" (scheduled at 2023-08-08 05:58:55.371682+00:00) notificationworker stdout | 2023-08-08 05:58:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:58:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 58, 55, 372920), True, datetime.datetime(2023, 8, 8, 5, 58, 55, 372920), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:58:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:58:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:58:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:58:56,262 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:58:56,794 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} securityworker stdout | 2023-08-08 05:58:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:58:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:19.687363+00:00 (in 22.796030 seconds) securityworker stdout | 2023-08-08 05:58:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:26 UTC)" (scheduled at 2023-08-08 05:58:56.890741+00:00) securityworker stdout | 2023-08-08 05:58:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:58:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:58:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:58:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:58:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:58:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:58:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:58:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:58:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:58:56,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:58:56,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:58:56,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:58:56,902 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:58:56,902 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:58:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:58:56,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:58:56,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:58:56,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:26 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 05:58:56,935 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 05:58:57,036 [199] [DEBUG] [app] Starting request: urn:request:164434a5-1d7e-448e-9089-b0d9960da757 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:57,036 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,036 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,037 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:57,043 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:57,043 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:57,043 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,043 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:57,043 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,043 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:57,043 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:57,043 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:57,046 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:57,052 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:57,053 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:57,057 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:57,061 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,061 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,061 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,062 [199] [DEBUG] [app] Ending request: urn:request:164434a5-1d7e-448e-9089-b0d9960da757 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'endpoint': 'api.repositorybuildstatus', 'request_id': 'urn:request:164434a5-1d7e-448e-9089-b0d9960da757', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:57,063 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:57 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/2.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 914 0.029) gunicorn-web stdout | 2023-08-08 05:58:57,063 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:57 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/1.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 05:58:57,231 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:58:57,255 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:58:57,267 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:58:57,279 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:58:57,282 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 05:58:57,386 [200] [DEBUG] [app] Starting request: urn:request:3f6947a9-15b5-41ee-8fad-4237984bbc71 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:58:57,386 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,386 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,387 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:57,393 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:58:57,393 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:58:57,393 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,394 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:58:57,394 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,394 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:57,394 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:57,394 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:58:57,396 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:58:57,403 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:58:57,404 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:57,406 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:58:57,407 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:58:57,413 [200] [DEBUG] [app] Ending request: urn:request:3f6947a9-15b5-41ee-8fad-4237984bbc71 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'endpoint': 'api.repositorybuildlogs', 'request_id': 'urn:request:3f6947a9-15b5-41ee-8fad-4237984bbc71', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs', 'parameters': {'start': '1'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:58:57,414 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:58:57,414 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:58:57 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/1.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:58:57 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/2.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 446 0.029) buildlogsarchiver stdout | 2023-08-08 05:58:58,177 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:58:58,366 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 05:58:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:58:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:09.417660+00:00 (in 9.999470 seconds) gcworker stdout | 2023-08-08 05:58:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:09 UTC)" (scheduled at 2023-08-08 05:58:59.417660+00:00) gcworker stdout | 2023-08-08 05:58:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264739418, 1209600, True, 3, 500, 1, 0]) securityworker stdout | 2023-08-08 05:58:59,427 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gcworker stdout | 2023-08-08 05:58:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:58:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:58:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:09 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:59:00,216 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:59:00,319 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:59:00,815 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 05:59:01,074 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:59:01,084 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:59:01,084 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:59:01,089 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:59:01,352 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:59:01,627 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:59:01,952 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 05:59:02,179 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:59:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:05.371682+00:00 (in 3.192507 seconds) notificationworker stdout | 2023-08-08 05:59:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:02 UTC)" (scheduled at 2023-08-08 05:59:02.178530+00:00) notificationworker stdout | 2023-08-08 05:59:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 05:59:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:59:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:59:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:12.293757+00:00 (in 9.999393 seconds) chunkcleanupworker stdout | 2023-08-08 05:59:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:12 UTC)" (scheduled at 2023-08-08 05:59:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:59:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:59:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 2, 294954), True, datetime.datetime(2023, 8, 8, 5, 59, 2, 294954), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:59:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:59:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:59:03,053 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:59:03,058 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:59:03,061 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:59:03,065 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:59:03,068 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:59:03,071 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:59:03,075 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:59:03,078 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:59:03,081 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-web stdout | 2023-08-08 05:59:03,492 [197] [DEBUG] [app] Starting request: urn:request:b95147c5-75e1-4568-a48f-ef0e11635415 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:59:03,492 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,492 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,493 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:03,500 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:59:03,500 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:59:03,500 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,500 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:59:03,500 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,500 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:03,500 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:03,500 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:03,502 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:59:03,509 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:59:03,510 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:03,513 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:03,518 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,518 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,518 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,519 [197] [DEBUG] [app] Ending request: urn:request:b95147c5-75e1-4568-a48f-ef0e11635415 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'endpoint': 'api.repositorybuildstatus', 'request_id': 'urn:request:b95147c5-75e1-4568-a48f-ef0e11635415', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:59:03,519 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:03,520 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:59:03 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/1.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:59:03 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/2.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 913 0.030) gunicorn-web stdout | 2023-08-08 05:59:03,841 [198] [DEBUG] [app] Starting request: urn:request:41d4fefe-19ab-47a1-bef9-d21ba87bee94 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:59:03,842 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,842 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,843 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:03,849 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:59:03,849 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:59:03,849 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,850 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:59:03,850 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,850 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:03,850 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:03,850 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:03,852 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:59:03,858 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:59:03,859 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:03,861 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:03,863 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:03,869 [198] [DEBUG] [app] Ending request: urn:request:41d4fefe-19ab-47a1-bef9-d21ba87bee94 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'endpoint': 'api.repositorybuildlogs', 'request_id': 'urn:request:41d4fefe-19ab-47a1-bef9-d21ba87bee94', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs', 'parameters': {'start': '1'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:59:03,869 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:59:03 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/2.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 446 0.030) gunicorn-web stdout | 2023-08-08 05:59:03,870 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:59:03 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/1.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" repositorygcworker stdout | 2023-08-08 05:59:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:59:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:09.793658+00:00 (in 5.605698 seconds) repositorygcworker stdout | 2023-08-08 05:59:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:04 UTC)" (scheduled at 2023-08-08 05:59:04.187429+00:00) repositorygcworker stdout | 2023-08-08 05:59:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 05:59:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:59:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:59:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:15.371682+00:00 (in 9.999473 seconds) notificationworker stdout | 2023-08-08 05:59:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:15 UTC)" (scheduled at 2023-08-08 05:59:05.371682+00:00) notificationworker stdout | 2023-08-08 05:59:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:59:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 5, 372777), True, datetime.datetime(2023, 8, 8, 5, 59, 5, 372777), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:59:05,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:59:05,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:59:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:59:07,178 [197] [DEBUG] [app] Starting request: urn:request:ad6841fe-d703-477b-9111-f2bd3d7cd697 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:59:07,179 [198] [DEBUG] [app] Starting request: urn:request:1278710a-677a-42f1-8578-cf27d86e75e7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:59:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:07,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:59:07,184 [206] [DEBUG] [app] Starting request: urn:request:1b1ded8c-9f8b-46a7-b1bb-54459cb28dc0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:07,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:59:07,185 [206] [DEBUG] [app] Ending request: urn:request:1b1ded8c-9f8b-46a7-b1bb-54459cb28dc0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1b1ded8c-9f8b-46a7-b1bb-54459cb28dc0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:59:07,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:59:07,185 [203] [DEBUG] [app] Starting request: urn:request:f3c3f42c-ab60-4017-bdb7-6ea92cef28e0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:59:07,186 [203] [DEBUG] [app] Ending request: urn:request:f3c3f42c-ab60-4017-bdb7-6ea92cef28e0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f3c3f42c-ab60-4017-bdb7-6ea92cef28e0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:59:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 05:59:07,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:07,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:07,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:07,191 [197] [DEBUG] [app] Starting request: urn:request:1b753b07-adec-448d-9603-b565dacfe99f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:07,191 [200] [DEBUG] [app] Starting request: urn:request:348103da-eb3e-465b-be4f-5b36dea7d621 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:07,191 [197] [DEBUG] [app] Ending request: urn:request:1b753b07-adec-448d-9603-b565dacfe99f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1b753b07-adec-448d-9603-b565dacfe99f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:59:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:59:07,192 [200] [DEBUG] [app] Ending request: urn:request:348103da-eb3e-465b-be4f-5b36dea7d621 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:348103da-eb3e-465b-be4f-5b36dea7d621', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:59:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:59:07,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:59:07,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:59:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:59:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 05:59:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:37.197412+00:00 (in 29.999519 seconds) buildlogsarchiver stdout | 2023-08-08 05:59:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:37 UTC)" (scheduled at 2023-08-08 05:59:07.197412+00:00) gunicorn-web stdout | 2023-08-08 05:59:07,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:59:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 59, 7, 198329), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:07,202 [197] [DEBUG] [app] Ending request: urn:request:ad6841fe-d703-477b-9111-f2bd3d7cd697 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ad6841fe-d703-477b-9111-f2bd3d7cd697', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:59:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:59:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:59:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:59:07,204 [198] [DEBUG] [app] Ending request: urn:request:1278710a-677a-42f1-8578-cf27d86e75e7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1278710a-677a-42f1-8578-cf27d86e75e7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:59:07,204 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:59:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:59:07,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:59:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:59:07,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:59:07,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:59:07,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:37 UTC)" executed successfully gcworker stdout | 2023-08-08 05:59:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:59:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:19.417660+00:00 (in 9.999489 seconds) gcworker stdout | 2023-08-08 05:59:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:19 UTC)" (scheduled at 2023-08-08 05:59:09.417660+00:00) gcworker stdout | 2023-08-08 05:59:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264749418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:59:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:59:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:59:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 05:59:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:59:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:19.187230+00:00 (in 9.393013 seconds) repositorygcworker stdout | 2023-08-08 05:59:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:09 UTC)" (scheduled at 2023-08-08 05:59:09.793658+00:00) repositorygcworker stdout | 2023-08-08 05:59:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 05:59:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 9, 794841), True, datetime.datetime(2023, 8, 8, 5, 59, 9, 794841), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 05:59:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 05:59:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:59:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:09 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:59:10,040 [197] [DEBUG] [app] Starting request: urn:request:1f71511b-3fcc-4960-b7a0-b26107231947 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:59:10,041 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,041 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,041 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:10,048 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:59:10,048 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:59:10,048 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,048 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:59:10,048 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,048 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:10,048 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:10,048 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:10,051 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:59:10,057 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:59:10,058 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:10,062 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:10,067 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,067 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,067 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,068 [197] [DEBUG] [app] Ending request: urn:request:1f71511b-3fcc-4960-b7a0-b26107231947 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'endpoint': 'api.repositorybuildstatus', 'request_id': 'urn:request:1f71511b-3fcc-4960-b7a0-b26107231947', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:59:10,068 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:10,069 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:59:10 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/1.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:59:10 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/2.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 913 0.030) gunicorn-web stdout | 2023-08-08 05:59:10,451 [199] [DEBUG] [app] Starting request: urn:request:0f7836cc-333c-4022-82c9-40bb5ede553f (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:59:10,451 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,451 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,452 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:10,458 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:59:10,458 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:59:10,458 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,458 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:59:10,458 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,458 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:10,458 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:10,458 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:10,461 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:59:10,467 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:59:10,468 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:10,470 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:10,472 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:10,477 [199] [DEBUG] [app] Ending request: urn:request:0f7836cc-333c-4022-82c9-40bb5ede553f (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'endpoint': 'api.repositorybuildlogs', 'request_id': 'urn:request:0f7836cc-333c-4022-82c9-40bb5ede553f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs', 'parameters': {'start': '1'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:59:10,478 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:59:10 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/2.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 446 0.029) gunicorn-web stdout | 2023-08-08 05:59:10,478 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:59:10 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/1.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" builder stdout | 2023-08-08 05:59:11,099 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:59:11,108 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:59:11,108 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:59:11,113 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:59:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:22.293757+00:00 (in 9.999327 seconds) chunkcleanupworker stdout | 2023-08-08 05:59:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:22 UTC)" (scheduled at 2023-08-08 05:59:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:59:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:59:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 12, 294965), True, datetime.datetime(2023, 8, 8, 5, 59, 12, 294965), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:59:12,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:59:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 05:59:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:59:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:54.284093+00:00 (in 41.186158 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:59:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:13 UTC)" (scheduled at 2023-08-08 05:59:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:59:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 05:59:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 13, 98296), True, datetime.datetime(2023, 8, 8, 5, 59, 13, 98296), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 05:59:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 05:59:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:59:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:59:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:59:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:25.371682+00:00 (in 9.999479 seconds) notificationworker stdout | 2023-08-08 05:59:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:25 UTC)" (scheduled at 2023-08-08 05:59:15.371682+00:00) notificationworker stdout | 2023-08-08 05:59:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:59:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 15, 372715), True, datetime.datetime(2023, 8, 8, 5, 59, 15, 372715), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:59:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:59:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:59:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:59:17,417 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:59:18,655 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} repositorygcworker stdout | 2023-08-08 05:59:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 05:59:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:04.187429+00:00 (in 44.999657 seconds) repositorygcworker stdout | 2023-08-08 05:59:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:04:19 UTC)" (scheduled at 2023-08-08 05:59:19.187230+00:00) repositorygcworker stdout | 2023-08-08 05:59:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 59, 19, 188240), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:59:19,195 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 59, 19, 188240), True, datetime.datetime(2023, 8, 8, 5, 59, 19, 188240), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:59:19,197 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 59, 19, 188240), True, datetime.datetime(2023, 8, 8, 5, 59, 19, 188240), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 5, 59, 19, 188240), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 05:59:19,198 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 05:59:19,198 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:04:19 UTC)" executed successfully gcworker stdout | 2023-08-08 05:59:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:59:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:29.417660+00:00 (in 9.999495 seconds) gcworker stdout | 2023-08-08 05:59:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:29 UTC)" (scheduled at 2023-08-08 05:59:19.417660+00:00) gcworker stdout | 2023-08-08 05:59:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264759418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:59:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:59:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:59:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:29 UTC)" executed successfully securityworker stdout | 2023-08-08 05:59:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:59:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:26.890741+00:00 (in 7.202849 seconds) securityworker stdout | 2023-08-08 05:59:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:49 UTC)" (scheduled at 2023-08-08 05:59:19.687363+00:00) securityworker stdout | 2023-08-08 05:59:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:59:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:59:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:59:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:59:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:59:19,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:59:19,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:59:19,792 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 05:59:20,223 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:59:21,123 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:59:21,132 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:59:21,132 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:59:21,136 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:22,180 [197] [DEBUG] [app] Starting request: urn:request:5efa478a-ffdf-4c93-9cc8-a724fae0a9da (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:59:22,180 [200] [DEBUG] [app] Starting request: urn:request:6b303f83-4dcd-45fa-8791-d53679dc8272 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:59:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:22,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:22,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:22,186 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:59:22,186 [207] [DEBUG] [app] Starting request: urn:request:df343866-b8a3-4a77-8e4a-fad064e30128 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:59:22,187 [209] [DEBUG] [app] Starting request: urn:request:e6db9560-10f2-4caa-ac3f-afacd1715b04 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:59:22,187 [207] [DEBUG] [app] Ending request: urn:request:df343866-b8a3-4a77-8e4a-fad064e30128 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:df343866-b8a3-4a77-8e4a-fad064e30128', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:59:22,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:59:22,187 [209] [DEBUG] [app] Ending request: urn:request:e6db9560-10f2-4caa-ac3f-afacd1715b04 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e6db9560-10f2-4caa-ac3f-afacd1715b04', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:59:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:59:22,188 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 05:59:22,188 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:22,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:22,190 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:22,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:22,192 [197] [DEBUG] [app] Starting request: urn:request:2f8c30c9-2bb4-4cee-a5c8-9547f046bcd4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:22,193 [197] [DEBUG] [app] Ending request: urn:request:2f8c30c9-2bb4-4cee-a5c8-9547f046bcd4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2f8c30c9-2bb4-4cee-a5c8-9547f046bcd4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:59:22,193 [199] [DEBUG] [app] Starting request: urn:request:3d012453-aceb-42f0-b6e7-fcea3b84d71f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:22,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:59:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:22,194 [199] [DEBUG] [app] Ending request: urn:request:3d012453-aceb-42f0-b6e7-fcea3b84d71f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3d012453-aceb-42f0-b6e7-fcea3b84d71f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 05:59:22,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:59:22,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:59:22,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:59:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:59:22,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:59:22,204 [197] [DEBUG] [app] Ending request: urn:request:5efa478a-ffdf-4c93-9cc8-a724fae0a9da (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5efa478a-ffdf-4c93-9cc8-a724fae0a9da', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:59:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:59:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:59:22,204 [200] [DEBUG] [app] Ending request: urn:request:6b303f83-4dcd-45fa-8791-d53679dc8272 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6b303f83-4dcd-45fa-8791-d53679dc8272', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:59:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 05:59:22,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:59:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:59:22,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:59:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 05:59:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:59:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:28.874906+00:00 (in 6.580444 seconds) chunkcleanupworker stdout | 2023-08-08 05:59:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:32 UTC)" (scheduled at 2023-08-08 05:59:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:59:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:59:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 22, 294954), True, datetime.datetime(2023, 8, 8, 5, 59, 22, 294954), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:59:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:59:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:32 UTC)" executed successfully gcworker stdout | 2023-08-08 05:59:22,979 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 05:59:24,816 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} gunicorn-secscan stdout | 2023-08-08 05:59:24,827 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:59:24,833 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:59:24,836 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} repositorygcworker stdout | 2023-08-08 05:59:24,898 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:59:25,195 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:59:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:59:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:35.371682+00:00 (in 9.999480 seconds) notificationworker stdout | 2023-08-08 05:59:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:35 UTC)" (scheduled at 2023-08-08 05:59:25.371682+00:00) notificationworker stdout | 2023-08-08 05:59:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:59:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 25, 372714), True, datetime.datetime(2023, 8, 8, 5, 59, 25, 372714), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:59:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:59:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:59:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:59:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:59:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:47.398551+00:00 (in 21.883635 seconds) exportactionlogsworker stdout | 2023-08-08 05:59:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:25 UTC)" (scheduled at 2023-08-08 05:59:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 05:59:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 05:59:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:59:26,299 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:59:26,831 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} securityworker stdout | 2023-08-08 05:59:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:59:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:49.687363+00:00 (in 22.796127 seconds) securityworker stdout | 2023-08-08 05:59:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:56 UTC)" (scheduled at 2023-08-08 05:59:26.890741+00:00) securityworker stdout | 2023-08-08 05:59:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:59:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:59:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:59:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:59:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:59:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:59:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:59:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:59:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:59:26,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:59:26,902 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:59:26,902 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:59:26,902 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:59:26,902 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:59:26,902 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:59:26,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:59:26,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:59:26,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 05:59:56 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 05:59:26,950 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 05:59:27,241 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:59:27,266 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:59:27,279 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:59:27,292 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:59:27,295 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:59:28,213 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:59:28,402 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 05:59:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:59:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:32.293757+00:00 (in 3.418312 seconds) chunkcleanupworker stdout | 2023-08-08 05:59:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:28 UTC)" (scheduled at 2023-08-08 05:59:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 05:59:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 05:59:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:28 UTC)" executed successfully gcworker stdout | 2023-08-08 05:59:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:59:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:39.417660+00:00 (in 9.999493 seconds) gcworker stdout | 2023-08-08 05:59:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:39 UTC)" (scheduled at 2023-08-08 05:59:29.417660+00:00) gcworker stdout | 2023-08-08 05:59:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264769418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:59:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:59:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:59:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:39 UTC)" executed successfully securityworker stdout | 2023-08-08 05:59:29,454 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 05:59:30,253 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 05:59:30,347 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 05:59:30,847 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 05:59:31,146 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:59:31,155 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:59:31,156 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:59:31,160 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 05:59:31,389 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 05:59:31,654 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 05:59:31,989 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 05:59:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:59:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:42.293757+00:00 (in 9.999457 seconds) chunkcleanupworker stdout | 2023-08-08 05:59:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:42 UTC)" (scheduled at 2023-08-08 05:59:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:59:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:59:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 32, 294782), True, datetime.datetime(2023, 8, 8, 5, 59, 32, 294782), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:59:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:59:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 05:59:33,061 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 05:59:33,066 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 05:59:33,070 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 05:59:33,073 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 05:59:33,076 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 05:59:33,082 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 05:59:33,086 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 05:59:33,089 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 05:59:33,092 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 05:59:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:59:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:45.371682+00:00 (in 9.999452 seconds) notificationworker stdout | 2023-08-08 05:59:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:45 UTC)" (scheduled at 2023-08-08 05:59:35.371682+00:00) notificationworker stdout | 2023-08-08 05:59:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:59:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 35, 372606), True, datetime.datetime(2023, 8, 8, 5, 59, 35, 372606), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:59:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:59:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:59:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:59:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:59:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:39.579576+00:00 (in 2.999190 seconds) namespacegcworker stdout | 2023-08-08 05:59:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:36 UTC)" (scheduled at 2023-08-08 05:59:36.579841+00:00) namespacegcworker stdout | 2023-08-08 05:59:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 05:59:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:59:37,178 [199] [DEBUG] [app] Starting request: urn:request:9ee6d632-ffb5-4b06-bcfa-ae0e027e7b5d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:59:37,178 [197] [DEBUG] [app] Starting request: urn:request:d2c2c088-9958-4d04-9eb6-cd11758e716c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:59:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:37,182 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:59:37,184 [210] [DEBUG] [app] Starting request: urn:request:8929f640-e6c4-4cdd-b334-9ade42240366 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:59:37,185 [210] [DEBUG] [app] Ending request: urn:request:8929f640-e6c4-4cdd-b334-9ade42240366 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8929f640-e6c4-4cdd-b334-9ade42240366', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 05:59:37,184 [209] [DEBUG] [app] Starting request: urn:request:cab76738-c7d5-4f95-9952-4438dff173e0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:59:37,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:59:37,185 [209] [DEBUG] [app] Ending request: urn:request:cab76738-c7d5-4f95-9952-4438dff173e0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:cab76738-c7d5-4f95-9952-4438dff173e0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:59:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:59:37,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:59:37,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:37,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:37,190 [200] [DEBUG] [app] Starting request: urn:request:05b028f5-e782-41e7-98ff-16bfcb5fed74 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:37,191 [198] [DEBUG] [app] Starting request: urn:request:2e8ee07c-5e9b-4d45-ac30-fa251cc60eae (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:37,191 [200] [DEBUG] [app] Ending request: urn:request:05b028f5-e782-41e7-98ff-16bfcb5fed74 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:05b028f5-e782-41e7-98ff-16bfcb5fed74', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:59:37,191 [198] [DEBUG] [app] Ending request: urn:request:2e8ee07c-5e9b-4d45-ac30-fa251cc60eae (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2e8ee07c-5e9b-4d45-ac30-fa251cc60eae', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:59:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 05:59:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:59:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 05:59:37,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:59:37,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:59:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:59:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 05:59:37,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 05:59:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:07.197412+00:00 (in 29.999514 seconds) buildlogsarchiver stdout | 2023-08-08 05:59:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:07 UTC)" (scheduled at 2023-08-08 05:59:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 05:59:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 5, 59, 37, 198427), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:37,202 [197] [DEBUG] [app] Ending request: urn:request:d2c2c088-9958-4d04-9eb6-cd11758e716c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d2c2c088-9958-4d04-9eb6-cd11758e716c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:59:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:59:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:59:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 05:59:37,203 [199] [DEBUG] [app] Ending request: urn:request:9ee6d632-ffb5-4b06-bcfa-ae0e027e7b5d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9ee6d632-ffb5-4b06-bcfa-ae0e027e7b5d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:59:37,203 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:59:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 05:59:37,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:59:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 05:59:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 05:59:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 05:59:37,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 05:59:38,507 [197] [DEBUG] [app] Starting request: urn:request:5eb8762e-cc75-4626-831d-5e3413bda583 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:59:38,507 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,507 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,509 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:38,516 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:59:38,516 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:59:38,516 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,516 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:59:38,516 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,516 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:38,516 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:38,516 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:38,519 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:59:38,525 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:59:38,526 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:38,530 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:38,534 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,534 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,534 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,535 [197] [DEBUG] [app] Ending request: urn:request:5eb8762e-cc75-4626-831d-5e3413bda583 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'endpoint': 'api.repositorybuildstatus', 'request_id': 'urn:request:5eb8762e-cc75-4626-831d-5e3413bda583', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:59:38,536 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:38,536 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:59:38 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/1.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:59:38 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/2.0" 200 2089 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 913 0.032) gunicorn-web stdout | 2023-08-08 05:59:38,920 [200] [DEBUG] [app] Starting request: urn:request:6e8da384-d470-464b-8a31-9d44940ed43c (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 05:59:38,921 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,921 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,921 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:38,928 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 05:59:38,929 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 05:59:38,929 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,929 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 05:59:38,929 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,929 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:38,929 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:38,929 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 05:59:38,932 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 05:59:38,937 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 05:59:38,938 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:38,940 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 05:59:38,942 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 05:59:38,948 [200] [DEBUG] [app] Ending request: urn:request:6e8da384-d470-464b-8a31-9d44940ed43c (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'endpoint': 'api.repositorybuildlogs', 'request_id': 'urn:request:6e8da384-d470-464b-8a31-9d44940ed43c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs', 'parameters': {'start': '1'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 05:59:38,948 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:05:59:38 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/2.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 447 0.029) gunicorn-web stdout | 2023-08-08 05:59:38,949 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:05:59:38 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/1.0" 200 37 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gcworker stdout | 2023-08-08 05:59:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:59:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:49.417660+00:00 (in 9.999486 seconds) gcworker stdout | 2023-08-08 05:59:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:49 UTC)" (scheduled at 2023-08-08 05:59:39.417660+00:00) gcworker stdout | 2023-08-08 05:59:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264779418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:59:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:59:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:59:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:59:39,579 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:59:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:54.393627+00:00 (in 14.813504 seconds) namespacegcworker stdout | 2023-08-08 05:59:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:04:39 UTC)" (scheduled at 2023-08-08 05:59:39.579576+00:00) namespacegcworker stdout | 2023-08-08 05:59:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 59, 39, 580458), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:59:39,588 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 59, 39, 580458), True, datetime.datetime(2023, 8, 8, 5, 59, 39, 580458), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:59:39,590 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 59, 39, 580458), True, datetime.datetime(2023, 8, 8, 5, 59, 39, 580458), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 5, 59, 39, 580458), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 05:59:39,591 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:59:39,591 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:04:39 UTC)" executed successfully builder stdout | 2023-08-08 05:59:41,161 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:59:41,170 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:59:41,170 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:59:41,175 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:59:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:46.874674+00:00 (in 4.580360 seconds) chunkcleanupworker stdout | 2023-08-08 05:59:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:52 UTC)" (scheduled at 2023-08-08 05:59:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:59:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:59:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 42, 294729), True, datetime.datetime(2023, 8, 8, 5, 59, 42, 294729), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:59:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:59:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:59:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:59:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:55.371682+00:00 (in 9.999466 seconds) notificationworker stdout | 2023-08-08 05:59:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:55 UTC)" (scheduled at 2023-08-08 05:59:45.371682+00:00) notificationworker stdout | 2023-08-08 05:59:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:59:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 45, 372590), True, datetime.datetime(2023, 8, 8, 5, 59, 45, 372590), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:59:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:59:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:59:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 05:59:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:59:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:52.293757+00:00 (in 5.418565 seconds) chunkcleanupworker stdout | 2023-08-08 05:59:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:04:46 UTC)" (scheduled at 2023-08-08 05:59:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 05:59:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 5, 59, 46, 875643), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:59:46,883 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 59, 46, 875643), True, datetime.datetime(2023, 8, 8, 5, 59, 46, 875643), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:59:46,885 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 5, 59, 46, 875643), True, datetime.datetime(2023, 8, 8, 5, 59, 46, 875643), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 5, 59, 46, 875643), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 05:59:46,886 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:04:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 05:59:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 05:59:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:25.514341+00:00 (in 38.115262 seconds) exportactionlogsworker stdout | 2023-08-08 05:59:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:47 UTC)" (scheduled at 2023-08-08 05:59:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 05:59:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 05:59:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 47, 399530), True, datetime.datetime(2023, 8, 8, 5, 59, 47, 399530), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 05:59:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 05:59:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 05:59:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 05:59:47,425 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 05:59:48,693 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 05:59:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:59:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:59.417660+00:00 (in 9.999444 seconds) gcworker stdout | 2023-08-08 05:59:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:59 UTC)" (scheduled at 2023-08-08 05:59:49.417660+00:00) gcworker stdout | 2023-08-08 05:59:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264789418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:59:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:59:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:59:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 05:59:59 UTC)" executed successfully securityworker stdout | 2023-08-08 05:59:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:59:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 05:59:56.890741+00:00 (in 7.202844 seconds) securityworker stdout | 2023-08-08 05:59:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:19 UTC)" (scheduled at 2023-08-08 05:59:49.687363+00:00) securityworker stdout | 2023-08-08 05:59:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:59:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:59:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:59:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:59:49,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:59:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:59:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 05:59:49,801 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 05:59:50,259 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 05:59:51,175 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 05:59:51,184 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 05:59:51,184 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 05:59:51,189 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:52,179 [197] [DEBUG] [app] Starting request: urn:request:d2bda559-5566-4dd2-98c9-8f1e09cdb11f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:59:52,180 [200] [DEBUG] [app] Starting request: urn:request:fa72d622-d88f-4cbc-aee9-b61322f15a4b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 05:59:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:52,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 05:59:52,185 [203] [DEBUG] [app] Starting request: urn:request:36220dff-4479-4dda-82cb-c963896e7ce1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:59:52,186 [210] [DEBUG] [app] Starting request: urn:request:633d8e8b-f5c7-4f5b-86c1-8684ff2cd559 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 05:59:52,186 [203] [DEBUG] [app] Ending request: urn:request:36220dff-4479-4dda-82cb-c963896e7ce1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:36220dff-4479-4dda-82cb-c963896e7ce1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 05:59:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 05:59:52,186 [210] [DEBUG] [app] Ending request: urn:request:633d8e8b-f5c7-4f5b-86c1-8684ff2cd559 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:633d8e8b-f5c7-4f5b-86c1-8684ff2cd559', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:59:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 05:59:52,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 05:59:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:52,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 05:59:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:52,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 05:59:52,192 [199] [DEBUG] [app] Starting request: urn:request:a08ac28c-d5f3-4aaa-97ea-95c206f129a3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:52,192 [200] [DEBUG] [app] Starting request: urn:request:b2da2a8a-2c44-4680-8241-4a5f9949c107 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 05:59:52,193 [199] [DEBUG] [app] Ending request: urn:request:a08ac28c-d5f3-4aaa-97ea-95c206f129a3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a08ac28c-d5f3-4aaa-97ea-95c206f129a3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 05:59:52,193 [200] [DEBUG] [app] Ending request: urn:request:b2da2a8a-2c44-4680-8241-4a5f9949c107 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b2da2a8a-2c44-4680-8241-4a5f9949c107', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:05:59:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 05:59:52,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:59:52,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:05:59:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 05:59:52,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 05:59:52,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:59:52,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 05:59:52,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:59:52,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 05:59:52,204 [197] [DEBUG] [app] Ending request: urn:request:d2bda559-5566-4dd2-98c9-8f1e09cdb11f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d2bda559-5566-4dd2-98c9-8f1e09cdb11f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:59:52,204 [200] [DEBUG] [app] Ending request: urn:request:fa72d622-d88f-4cbc-aee9-b61322f15a4b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fa72d622-d88f-4cbc-aee9-b61322f15a4b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 05:59:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 05:59:52,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:59:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 05:59:52,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:05:59:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:59:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:05:59:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) chunkcleanupworker stdout | 2023-08-08 05:59:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 05:59:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:02.293757+00:00 (in 9.999445 seconds) chunkcleanupworker stdout | 2023-08-08 05:59:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:02 UTC)" (scheduled at 2023-08-08 05:59:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 05:59:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 05:59:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 52, 294922), True, datetime.datetime(2023, 8, 8, 5, 59, 52, 294922), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 05:59:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 05:59:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 05:59:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:02 UTC)" executed successfully gcworker stdout | 2023-08-08 05:59:52,993 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 05:59:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 05:59:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:13.097375+00:00 (in 18.812773 seconds) securityscanningnotificationworker stdout | 2023-08-08 05:59:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:54 UTC)" (scheduled at 2023-08-08 05:59:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 05:59:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 05:59:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 05:59:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 05:59:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:36.579841+00:00 (in 42.185660 seconds) namespacegcworker stdout | 2023-08-08 05:59:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:54 UTC)" (scheduled at 2023-08-08 05:59:54.393627+00:00) namespacegcworker stdout | 2023-08-08 05:59:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 05:59:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 54, 394873), True, datetime.datetime(2023, 8, 8, 5, 59, 54, 394873), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 05:59:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 05:59:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 05:59:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:00:54 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 05:59:54,823 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} gunicorn-secscan stdout | 2023-08-08 05:59:54,836 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 05:59:54,840 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 05:59:54,843 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} repositorygcworker stdout | 2023-08-08 05:59:54,921 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 05:59:55,208 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 05:59:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 05:59:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:02.178530+00:00 (in 6.806295 seconds) notificationworker stdout | 2023-08-08 05:59:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:05 UTC)" (scheduled at 2023-08-08 05:59:55.371682+00:00) notificationworker stdout | 2023-08-08 05:59:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 05:59:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 5, 59, 55, 372593), True, datetime.datetime(2023, 8, 8, 5, 59, 55, 372593), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 05:59:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 05:59:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 05:59:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 05:59:56,307 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 05:59:56,858 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} securityworker stdout | 2023-08-08 05:59:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 05:59:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:19.687363+00:00 (in 22.796056 seconds) securityworker stdout | 2023-08-08 05:59:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:26 UTC)" (scheduled at 2023-08-08 05:59:56.890741+00:00) securityworker stdout | 2023-08-08 05:59:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:59:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 05:59:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:59:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 05:59:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:59:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 05:59:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 05:59:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 05:59:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 05:59:56,903 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:59:56,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 05:59:56,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 05:59:56,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 05:59:56,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 05:59:56,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 05:59:56,904 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 05:59:56,904 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 05:59:56,904 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:26 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 05:59:56,977 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 05:59:57,249 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 05:59:57,280 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 05:59:57,290 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 05:59:57,305 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 05:59:57,307 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 05:59:58,251 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 05:59:58,435 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 05:59:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 05:59:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:09.417660+00:00 (in 9.999504 seconds) gcworker stdout | 2023-08-08 05:59:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:09 UTC)" (scheduled at 2023-08-08 05:59:59.417660+00:00) gcworker stdout | 2023-08-08 05:59:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264799418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 05:59:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 05:59:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 05:59:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:09 UTC)" executed successfully securityworker stdout | 2023-08-08 05:59:59,491 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:00:00,278 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:00:00,384 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:00:00,864 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 06:00:01,199 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:00:01,209 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:00:01,209 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:00:01,213 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:00:01,399 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:00:01,677 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:00:02,000 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:00:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:00:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:05.371682+00:00 (in 3.192620 seconds) notificationworker stdout | 2023-08-08 06:00:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:02 UTC)" (scheduled at 2023-08-08 06:00:02.178530+00:00) notificationworker stdout | 2023-08-08 06:00:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:00:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:00:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:00:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:12.293757+00:00 (in 9.999477 seconds) chunkcleanupworker stdout | 2023-08-08 06:00:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:12 UTC)" (scheduled at 2023-08-08 06:00:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:00:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:00:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 2, 294673), True, datetime.datetime(2023, 8, 8, 6, 0, 2, 294673), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:00:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:00:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:00:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:00:03,070 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:00:03,073 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:00:03,079 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:00:03,082 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:00:03,085 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:00:03,094 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:00:03,096 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:00:03,099 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:00:03,103 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:00:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:00:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:09.793658+00:00 (in 5.605677 seconds) repositorygcworker stdout | 2023-08-08 06:00:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:04 UTC)" (scheduled at 2023-08-08 06:00:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:00:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:00:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:00:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:00:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:15.371682+00:00 (in 9.999471 seconds) notificationworker stdout | 2023-08-08 06:00:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:15 UTC)" (scheduled at 2023-08-08 06:00:05.371682+00:00) notificationworker stdout | 2023-08-08 06:00:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:00:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 5, 372599), True, datetime.datetime(2023, 8, 8, 6, 0, 5, 372599), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:00:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:00:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:00:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:00:07,179 [199] [DEBUG] [app] Starting request: urn:request:8f04517e-3c0b-4578-a61c-19780e357d9e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:00:07,180 [197] [DEBUG] [app] Starting request: urn:request:9b307f0f-c230-4a91-ad24-1bb50aa1bda0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:00:07,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:00:07,184 [210] [DEBUG] [app] Starting request: urn:request:c5da0ab1-a0b6-474a-9547-a47aee229524 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:00:07,185 [209] [DEBUG] [app] Starting request: urn:request:37cb4633-899b-4288-874d-c16c7a662f71 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:00:07,185 [210] [DEBUG] [app] Ending request: urn:request:c5da0ab1-a0b6-474a-9547-a47aee229524 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c5da0ab1-a0b6-474a-9547-a47aee229524', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:00:07,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:00:07,185 [209] [DEBUG] [app] Ending request: urn:request:37cb4633-899b-4288-874d-c16c7a662f71 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:37cb4633-899b-4288-874d-c16c7a662f71', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:00:07,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:00:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:00:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:07,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:07,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:07,191 [198] [DEBUG] [app] Starting request: urn:request:c609912e-c4c6-4806-9df7-c0b5b309deca (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:00:07,191 [198] [DEBUG] [app] Ending request: urn:request:c609912e-c4c6-4806-9df7-c0b5b309deca (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c609912e-c4c6-4806-9df7-c0b5b309deca', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:00:07,191 [200] [DEBUG] [app] Starting request: urn:request:76db580a-5b26-4177-915e-e6b0ff23eb01 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:00:07,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:00:07,192 [200] [DEBUG] [app] Ending request: urn:request:76db580a-5b26-4177-915e-e6b0ff23eb01 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:76db580a-5b26-4177-915e-e6b0ff23eb01', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:00:07,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:00:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:07,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:00:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:00:07,198 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:00:07,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:00:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:00:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:37.197412+00:00 (in 29.999291 seconds) buildlogsarchiver stdout | 2023-08-08 06:00:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:37 UTC)" (scheduled at 2023-08-08 06:00:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:00:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 0, 7, 198624), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:00:07,203 [197] [DEBUG] [app] Ending request: urn:request:9b307f0f-c230-4a91-ad24-1bb50aa1bda0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9b307f0f-c230-4a91-ad24-1bb50aa1bda0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:00:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:07,203 [199] [DEBUG] [app] Ending request: urn:request:8f04517e-3c0b-4578-a61c-19780e357d9e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8f04517e-3c0b-4578-a61c-19780e357d9e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:00:07,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:00:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:00:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:00:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:00:07,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:00:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:00:07,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 06:00:07,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:00:07,205 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:00:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:00:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:19.417660+00:00 (in 9.999487 seconds) gcworker stdout | 2023-08-08 06:00:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:19 UTC)" (scheduled at 2023-08-08 06:00:09.417660+00:00) gcworker stdout | 2023-08-08 06:00:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264809418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:00:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:00:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:00:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:00:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:00:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:04.187429+00:00 (in 54.393225 seconds) repositorygcworker stdout | 2023-08-08 06:00:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:09 UTC)" (scheduled at 2023-08-08 06:00:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:00:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:00:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 9, 794741), True, datetime.datetime(2023, 8, 8, 6, 0, 9, 794741), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:00:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:00:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:00:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:09 UTC)" executed successfully servicekey stdout | 2023-08-08 06:00:10,374 [78] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2023-08-08 06:00:10,374 [78] [INFO] [apscheduler.executors.default] Running job "ServiceKeyWorker._refresh_service_key (trigger: interval[0:55:00], next run at: 2023-08-08 06:00:10 UTC)" (scheduled at 2023-08-08 06:00:10.373311+00:00) servicekey stdout | 2023-08-08 06:00:10,375 [78] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:55:10.373311+00:00 (in 3299.999051 seconds) servicekey stdout | 2023-08-08 06:00:10,375 [78] [DEBUG] [__main__] Starting automatic refresh of service key 2ppoNkYeQyfLVZsxGBppQUgq8FR_iG_gON3o5opcqjI to new expiration 2023-08-08 08:00:10.375036 servicekey stdout | 2023-08-08 06:00:10,376 [78] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE (("t1"."kid" = %s) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s))) LIMIT %s OFFSET %s', ['2ppoNkYeQyfLVZsxGBppQUgq8FR_iG_gON3o5opcqjI', datetime.datetime(2023, 8, 1, 6, 0, 10, 375709), None, 1, 0]) servicekey stdout | 2023-08-08 06:00:10,382 [78] [DEBUG] [peewee] ('UPDATE "servicekey" SET "name" = %s, "kid" = %s, "service" = %s, "jwk" = %s, "metadata" = %s, "created_date" = %s, "expiration_date" = %s, "rotation_duration" = %s, "approval_id" = %s WHERE ("servicekey"."id" = %s)', ['https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com:443', '2ppoNkYeQyfLVZsxGBppQUgq8FR_iG_gON3o5opcqjI', 'quay', '{"n": "uwp0woMSUn8hKqxOuDC0949VejY4elekTyCG24UtG-YcLEA0jmA4S_mCCJcBHpSNm5aCKvyDYviNbBd0jcRdgUFbT5YeCgfDN32xgYpFbPukQQUv_stOrxD2-8rBIU5gaz29BsbGyQSlwkFnV1r4TE6_xavRwNaVPcWd60qLrpLLY0ovlZ8S2aq-Xq56OR-ZGGMy4jMa3HkuUz3YOx0Yxr3CiZz476RI01ekqBEt7YFi4UUJCVokEc3Eq-WnpqrWAQykn6SnWnR2kVKczM1HRKiGQdpBvUJ_M5mWxzlY3r4PavyXfMvSDx7KK38qss8rqdP3AU4NlNEp1Z9nJUTvzQ", "e": "AQAB", "kty": "RSA", "kid": "2ppoNkYeQyfLVZsxGBppQUgq8FR_iG_gON3o5opcqjI"}', '{"created_by": "CLI tool"}', datetime.datetime(2023, 8, 8, 5, 45, 28, 250518), datetime.datetime(2023, 8, 8, 8, 0, 10, 375036), None, 8, 8]) servicekey stdout | 2023-08-08 06:00:10,386 [78] [DEBUG] [__main__] Finished automatic refresh of service key 2ppoNkYeQyfLVZsxGBppQUgq8FR_iG_gON3o5opcqjI with new expiration 2023-08-08 08:00:10.375036 servicekey stdout | 2023-08-08 06:00:10,386 [78] [DEBUG] [data.database] Disconnecting from database. servicekey stdout | 2023-08-08 06:00:10,386 [78] [INFO] [apscheduler.executors.default] Job "ServiceKeyWorker._refresh_service_key (trigger: interval[0:55:00], next run at: 2023-08-08 06:55:10 UTC)" executed successfully builder stdout | 2023-08-08 06:00:11,214 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:00:11,225 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:00:11,225 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:00:11,230 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:00:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:00:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:22.293757+00:00 (in 9.999474 seconds) chunkcleanupworker stdout | 2023-08-08 06:00:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:22 UTC)" (scheduled at 2023-08-08 06:00:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:00:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:00:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 12, 294676), True, datetime.datetime(2023, 8, 8, 6, 0, 12, 294676), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:00:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:00:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:00:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:00:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:00:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:54.284093+00:00 (in 41.186156 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:00:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:13 UTC)" (scheduled at 2023-08-08 06:00:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:00:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:00:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 13, 98294), True, datetime.datetime(2023, 8, 8, 6, 0, 13, 98294), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:00:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:00:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:00:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:00:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:00:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:25.371682+00:00 (in 9.999464 seconds) notificationworker stdout | 2023-08-08 06:00:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:25 UTC)" (scheduled at 2023-08-08 06:00:15.371682+00:00) notificationworker stdout | 2023-08-08 06:00:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:00:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 15, 372589), True, datetime.datetime(2023, 8, 8, 6, 0, 15, 372589), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:00:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:00:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:00:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:00:17,458 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:00:18,711 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:00:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:00:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:29.417660+00:00 (in 9.999452 seconds) gcworker stdout | 2023-08-08 06:00:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:29 UTC)" (scheduled at 2023-08-08 06:00:19.417660+00:00) gcworker stdout | 2023-08-08 06:00:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264819418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:00:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:00:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:00:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:00:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:00:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:26.890741+00:00 (in 7.202699 seconds) securityworker stdout | 2023-08-08 06:00:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:49 UTC)" (scheduled at 2023-08-08 06:00:19.687363+00:00) securityworker stdout | 2023-08-08 06:00:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:00:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:00:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:00:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:00:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:00:19,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:00:19,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:00:19,839 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:00:20,297 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:00:21,238 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:00:21,248 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:00:21,248 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:00:21,253 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:22,179 [197] [DEBUG] [app] Starting request: urn:request:08d1e76a-4f96-4135-8267-f8783d498666 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:00:22,179 [198] [DEBUG] [app] Starting request: urn:request:85a788fc-1e51-4daa-9421-554bcc4a3011 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:00:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:22,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:22,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:00:22,184 [203] [DEBUG] [app] Starting request: urn:request:d4d21ca1-cb6a-494c-963c-04c4cca2135a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:00:22,185 [203] [DEBUG] [app] Ending request: urn:request:d4d21ca1-cb6a-494c-963c-04c4cca2135a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d4d21ca1-cb6a-494c-963c-04c4cca2135a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:00:22,185 [210] [DEBUG] [app] Starting request: urn:request:6697442c-964c-4e69-b29f-4d1bf06df838 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:00:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:00:22,185 [210] [DEBUG] [app] Ending request: urn:request:6697442c-964c-4e69-b29f-4d1bf06df838 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6697442c-964c-4e69-b29f-4d1bf06df838', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:00:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:00:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:00:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:22,190 [198] [DEBUG] [app] Starting request: urn:request:a0d5a6b6-4b7b-444e-a500-6ea4c144198d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:00:22,190 [198] [DEBUG] [app] Ending request: urn:request:a0d5a6b6-4b7b-444e-a500-6ea4c144198d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a0d5a6b6-4b7b-444e-a500-6ea4c144198d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:00:22,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:00:22,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:22,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:22,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:00:22,192 [200] [DEBUG] [app] Starting request: urn:request:7aa24c0b-829a-4e0f-af52-2cf825e807da (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:00:22,193 [200] [DEBUG] [app] Ending request: urn:request:7aa24c0b-829a-4e0f-af52-2cf825e807da (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7aa24c0b-829a-4e0f-af52-2cf825e807da', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:00:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:00:22,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:22,194 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:00:22,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:00:22,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:00:22,201 [197] [DEBUG] [app] Ending request: urn:request:08d1e76a-4f96-4135-8267-f8783d498666 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:08d1e76a-4f96-4135-8267-f8783d498666', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:00:22,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:22,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:00:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:00:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:00:22,204 [198] [DEBUG] [app] Ending request: urn:request:85a788fc-1e51-4daa-9421-554bcc4a3011 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:85a788fc-1e51-4daa-9421-554bcc4a3011', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:00:22,204 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:22,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:00:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:00:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:00:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:00:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:28.874906+00:00 (in 6.580638 seconds) chunkcleanupworker stdout | 2023-08-08 06:00:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:32 UTC)" (scheduled at 2023-08-08 06:00:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:00:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:00:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 22, 294818), True, datetime.datetime(2023, 8, 8, 6, 0, 22, 294818), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:00:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:00:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:00:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:00:23,026 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} blobuploadcleanupworker stdout | 2023-08-08 06:00:24,838 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} gunicorn-secscan stdout | 2023-08-08 06:00:24,845 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:00:24,848 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:00:24,851 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} repositorygcworker stdout | 2023-08-08 06:00:24,958 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:00:25,218 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 06:00:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:00:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:35.371682+00:00 (in 9.999448 seconds) notificationworker stdout | 2023-08-08 06:00:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:35 UTC)" (scheduled at 2023-08-08 06:00:25.371682+00:00) notificationworker stdout | 2023-08-08 06:00:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:00:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 25, 372606), True, datetime.datetime(2023, 8, 8, 6, 0, 25, 372606), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:00:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:00:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:00:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:00:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:00:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:47.398551+00:00 (in 21.883609 seconds) exportactionlogsworker stdout | 2023-08-08 06:00:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:25 UTC)" (scheduled at 2023-08-08 06:00:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:00:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:00:25,516 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:00:26,344 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} queuecleanupworker stdout | 2023-08-08 06:00:26,873 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} securityworker stdout | 2023-08-08 06:00:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:00:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:49.687363+00:00 (in 22.796098 seconds) securityworker stdout | 2023-08-08 06:00:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:56 UTC)" (scheduled at 2023-08-08 06:00:26.890741+00:00) securityworker stdout | 2023-08-08 06:00:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:00:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:00:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:00:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:00:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:00:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:00:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:00:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:00:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:00:26,902 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:00:26,903 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:00:26,903 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:00:26,903 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:00:26,903 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:00:26,903 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:00:26,903 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:00:26,903 [77] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 40, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 254, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2023-08-08 06:00:26,903 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:00:56 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 06:00:26,997 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:00:27,258 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:00:27,292 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:00:27,301 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:00:27,319 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:00:27,322 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:00:28,286 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:00:28,456 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 06:00:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:00:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:32.293757+00:00 (in 3.418284 seconds) chunkcleanupworker stdout | 2023-08-08 06:00:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:28 UTC)" (scheduled at 2023-08-08 06:00:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:00:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:00:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:00:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:00:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:39.417660+00:00 (in 9.999467 seconds) gcworker stdout | 2023-08-08 06:00:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:39 UTC)" (scheduled at 2023-08-08 06:00:29.417660+00:00) gcworker stdout | 2023-08-08 06:00:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264829418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:00:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:00:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:00:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:00:29,527 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:00:30,315 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:00:30,418 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:00:30,902 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 06:00:31,264 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:00:31,273 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:00:31,274 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:00:31,278 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:00:31,437 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:00:31,697 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:00:32,039 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 06:00:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:00:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:42.293757+00:00 (in 9.999417 seconds) chunkcleanupworker stdout | 2023-08-08 06:00:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:42 UTC)" (scheduled at 2023-08-08 06:00:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:00:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:00:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 32, 294835), True, datetime.datetime(2023, 8, 8, 6, 0, 32, 294835), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:00:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:00:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:00:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:00:33,079 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:00:33,089 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:00:33,092 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:00:33,092 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:00:33,095 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:00:33,105 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:00:33,113 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:00:33,116 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:00:33,119 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:00:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:00:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:45.371682+00:00 (in 9.999477 seconds) notificationworker stdout | 2023-08-08 06:00:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:45 UTC)" (scheduled at 2023-08-08 06:00:35.371682+00:00) notificationworker stdout | 2023-08-08 06:00:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:00:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 35, 372573), True, datetime.datetime(2023, 8, 8, 6, 0, 35, 372573), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:00:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:00:35,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:00:35,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:00:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:00:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:54.393627+00:00 (in 17.813153 seconds) namespacegcworker stdout | 2023-08-08 06:00:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:36 UTC)" (scheduled at 2023-08-08 06:00:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:00:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:00:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:00:37,179 [200] [DEBUG] [app] Starting request: urn:request:f300fc71-df7c-4f5e-baa0-1199a1cc765f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:00:37,180 [197] [DEBUG] [app] Starting request: urn:request:409edb6e-7c8a-4b3a-bfc7-92740444d8af (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:00:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:00:37,185 [209] [DEBUG] [app] Starting request: urn:request:aa453968-5681-4390-8669-34beaf2c672c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:00:37,185 [207] [DEBUG] [app] Starting request: urn:request:f0ca46d6-63a6-4402-86c1-0658d47ecc51 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:00:37,185 [209] [DEBUG] [app] Ending request: urn:request:aa453968-5681-4390-8669-34beaf2c672c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:aa453968-5681-4390-8669-34beaf2c672c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:00:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:00:37,186 [207] [DEBUG] [app] Ending request: urn:request:f0ca46d6-63a6-4402-86c1-0658d47ecc51 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f0ca46d6-63a6-4402-86c1-0658d47ecc51', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:00:37,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 06:00:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:37,191 [200] [DEBUG] [app] Starting request: urn:request:d7fb6458-a9c5-4bf9-a8f9-7d3f9e39c1fa (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:00:37,192 [200] [DEBUG] [app] Ending request: urn:request:d7fb6458-a9c5-4bf9-a8f9-7d3f9e39c1fa (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d7fb6458-a9c5-4bf9-a8f9-7d3f9e39c1fa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:00:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:00:37,192 [199] [DEBUG] [app] Starting request: urn:request:b16eaf3c-c88d-4da4-88c3-c1bd14bf3ed5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:00:37,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:00:37,193 [199] [DEBUG] [app] Ending request: urn:request:b16eaf3c-c88d-4da4-88c3-c1bd14bf3ed5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b16eaf3c-c88d-4da4-88c3-c1bd14bf3ed5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:00:37,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:00:37,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:37,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:00:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:00:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:07.197412+00:00 (in 29.999532 seconds) buildlogsarchiver stdout | 2023-08-08 06:00:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:07 UTC)" (scheduled at 2023-08-08 06:00:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:00:37,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:00:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 0, 37, 198362), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:00:37,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:00:37,204 [200] [DEBUG] [app] Ending request: urn:request:f300fc71-df7c-4f5e-baa0-1199a1cc765f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f300fc71-df7c-4f5e-baa0-1199a1cc765f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:00:37,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:37,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:00:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:00:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:00:37,205 [197] [DEBUG] [app] Ending request: urn:request:409edb6e-7c8a-4b3a-bfc7-92740444d8af (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:409edb6e-7c8a-4b3a-bfc7-92740444d8af', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:00:37,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:00:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:00:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:00:37,205 [56] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2023-08-08 06:00:37,205 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:00:37,206 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:00:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:00:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:49.417660+00:00 (in 9.999494 seconds) gcworker stdout | 2023-08-08 06:00:39,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:49 UTC)" (scheduled at 2023-08-08 06:00:39.417660+00:00) gcworker stdout | 2023-08-08 06:00:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264839419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:00:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:00:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:00:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:49 UTC)" executed successfully builder stdout | 2023-08-08 06:00:41,281 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:00:41,291 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:00:41,291 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:00:41,295 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:00:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:00:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:52.293757+00:00 (in 9.999464 seconds) chunkcleanupworker stdout | 2023-08-08 06:00:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:52 UTC)" (scheduled at 2023-08-08 06:00:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:00:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:00:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 42, 294801), True, datetime.datetime(2023, 8, 8, 6, 0, 42, 294801), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:00:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:00:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:00:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:00:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:00:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:55.371682+00:00 (in 9.999468 seconds) notificationworker stdout | 2023-08-08 06:00:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:55 UTC)" (scheduled at 2023-08-08 06:00:45.371682+00:00) notificationworker stdout | 2023-08-08 06:00:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:00:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 45, 372567), True, datetime.datetime(2023, 8, 8, 6, 0, 45, 372567), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:00:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:00:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:00:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:00:47,399 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:00:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:25.514341+00:00 (in 38.115200 seconds) exportactionlogsworker stdout | 2023-08-08 06:00:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:47 UTC)" (scheduled at 2023-08-08 06:00:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:00:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:00:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 47, 399643), True, datetime.datetime(2023, 8, 8, 6, 0, 47, 399643), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:00:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:00:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:00:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:00:47,495 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:00:48,748 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:00:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:00:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:59.417660+00:00 (in 9.999461 seconds) gcworker stdout | 2023-08-08 06:00:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:59 UTC)" (scheduled at 2023-08-08 06:00:49.417660+00:00) gcworker stdout | 2023-08-08 06:00:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264849418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:00:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:00:49,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:00:49,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:00:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:00:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:00:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:00:56.890741+00:00 (in 7.202888 seconds) securityworker stdout | 2023-08-08 06:00:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:19 UTC)" (scheduled at 2023-08-08 06:00:49.687363+00:00) securityworker stdout | 2023-08-08 06:00:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:00:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:00:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:00:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:00:49,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:00:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:00:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:00:49,867 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:00:50,335 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:00:51,298 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:00:51,308 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:00:51,308 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:00:51,313 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:52,179 [200] [DEBUG] [app] Starting request: urn:request:347d81eb-252f-477d-bf27-fc7d028bc9c8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:00:52,180 [197] [DEBUG] [app] Starting request: urn:request:0ef44d07-7432-4a3c-ac87-9a03ac83bae3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:00:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:00:52,185 [209] [DEBUG] [app] Starting request: urn:request:0833e45a-dddc-4883-a545-addde13c4f5a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:00:52,185 [210] [DEBUG] [app] Starting request: urn:request:76c20187-a09a-4525-bc9e-add7863b3f4d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:00:52,186 [209] [DEBUG] [app] Ending request: urn:request:0833e45a-dddc-4883-a545-addde13c4f5a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0833e45a-dddc-4883-a545-addde13c4f5a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:00:52,186 [210] [DEBUG] [app] Ending request: urn:request:76c20187-a09a-4525-bc9e-add7863b3f4d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:76c20187-a09a-4525-bc9e-add7863b3f4d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:00:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:00:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:00:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:00:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:00:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:00:52,191 [197] [DEBUG] [app] Starting request: urn:request:a81089d3-4825-4e62-b5e3-0d21fe40519d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:00:52,192 [199] [DEBUG] [app] Starting request: urn:request:29935481-b646-43e9-a441-3599b921451e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:00:52,192 [197] [DEBUG] [app] Ending request: urn:request:a81089d3-4825-4e62-b5e3-0d21fe40519d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a81089d3-4825-4e62-b5e3-0d21fe40519d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:00:52,192 [199] [DEBUG] [app] Ending request: urn:request:29935481-b646-43e9-a441-3599b921451e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:29935481-b646-43e9-a441-3599b921451e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:00:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:00:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:00:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:00:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:00:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:00:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:00:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:00:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:00:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:00:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:00:52,203 [200] [DEBUG] [app] Ending request: urn:request:347d81eb-252f-477d-bf27-fc7d028bc9c8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:347d81eb-252f-477d-bf27-fc7d028bc9c8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:00:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:00:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:00:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:00:52,205 [197] [DEBUG] [app] Ending request: urn:request:0ef44d07-7432-4a3c-ac87-9a03ac83bae3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0ef44d07-7432-4a3c-ac87-9a03ac83bae3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:00:52,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:00:52,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:00:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:00:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:00:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:00:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:02.293757+00:00 (in 9.999472 seconds) chunkcleanupworker stdout | 2023-08-08 06:00:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:02 UTC)" (scheduled at 2023-08-08 06:00:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:00:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:00:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 52, 294794), True, datetime.datetime(2023, 8, 8, 6, 0, 52, 294794), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:00:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:00:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:00:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:00:53,064 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:00:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:00:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:13.097375+00:00 (in 18.812765 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:00:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:54 UTC)" (scheduled at 2023-08-08 06:00:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:00:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:00:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:00:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:00:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:36.579841+00:00 (in 42.185670 seconds) namespacegcworker stdout | 2023-08-08 06:00:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:54 UTC)" (scheduled at 2023-08-08 06:00:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:00:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:00:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 54, 394685), True, datetime.datetime(2023, 8, 8, 6, 0, 54, 394685), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:00:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:00:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:00:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:01:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:00:54,854 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:00:54,857 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:00:54,860 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:00:54,869 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:00:54,981 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:00:55,252 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 06:00:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:00:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:02.178530+00:00 (in 6.806340 seconds) notificationworker stdout | 2023-08-08 06:00:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:05 UTC)" (scheduled at 2023-08-08 06:00:55.371682+00:00) notificationworker stdout | 2023-08-08 06:00:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:00:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 0, 55, 372771), True, datetime.datetime(2023, 8, 8, 6, 0, 55, 372771), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:00:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:00:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:00:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:00:56,364 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:00:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:00:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:19.687363+00:00 (in 22.796102 seconds) securityworker stdout | 2023-08-08 06:00:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:26 UTC)" (scheduled at 2023-08-08 06:00:56.890741+00:00) securityworker stdout | 2023-08-08 06:00:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:00:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:00:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:00:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:00:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:00:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:00:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:00:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:00:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) queuecleanupworker stdout | 2023-08-08 06:00:56,899 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} securityworker stdout | 2023-08-08 06:00:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:00:56,905 [77] [INFO] [util.migrate.allocator] Found 1 candidates, processing block start: 1 end: 2 securityworker stdout | 2023-08-08 06:00:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "mediatype" AS "t1"', []) securityworker stdout | 2023-08-08 06:00:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) securityworker stdout | 2023-08-08 06:00:56,910 [77] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:e6b52c3b1f61dec2ad1aa5e78bc737bff4d1f8709cb246b31adb518ca6146886', 1, 0]) securityworker stdout | 2023-08-08 06:00:56,912 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['15e67204-2b26-4a6e-b19c-1fb64267922c']) securityworker stdout | 2023-08-08 06:00:56,913 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) securityworker stdout | 2023-08-08 06:00:56,915 [77] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens securityworker stdout | 2023-08-08 06:00:56,916 [77] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 securityworker stdout | 2023-08-08 06:00:56,987 [77] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 securityworker stdout | 2023-08-08 06:00:56,989 [77] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["GxWy0IboSKWB6lePthTy-w"], "expires_at": "2023-08-08T18:00:56.000000Z", "issued_at": "2023-08-08T06:00:56.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} securityworker stdout | 2023-08-08 06:00:56,991 [77] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 repositoryactioncounter stdout | 2023-08-08 06:00:57,009 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} securityworker stdout | 2023-08-08 06:00:57,120 [77] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "GET /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/e6/e6b52c3b1f61dec2ad1aa5e78bc737bff4d1f8709cb246b31adb518ca6146886 HTTP/1.1" 200 7826 securityworker stdout | 2023-08-08 06:00:57,121 [77] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/e6/e6b52c3b1f61dec2ad1aa5e78bc737bff4d1f8709cb246b31adb518ca6146886 -X GET -H "X-Auth-Token: gAAAAABk0doYxeZx..." securityworker stdout | 2023-08-08 06:00:57,121 [77] [DEBUG] [swiftclient] RESP STATUS: 200 OK securityworker stdout | 2023-08-08 06:00:57,121 [77] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'X-Object-Manifest': 'quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:54 GMT', 'X-Timestamp': '1691474453.08321', 'Accept-Ranges': 'bytes', 'Content-Length': '7826', 'Etag': '"475f7ef4e505457a607a51723c86698f"', 'X-Trans-Id': 'tx3779c30c752b4718ad0b9-0064d1da19', 'X-Openstack-Request-Id': 'tx3779c30c752b4718ad0b9-0064d1da19', 'Date': 'Tue, 08 Aug 2023 06:00:57 GMT'} securityworker stdout | 2023-08-08 06:00:57,123 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', ['sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 1, 0]) securityworker stdout | 2023-08-08 06:00:57,125 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', ['sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 1, 0]) securityworker stdout | 2023-08-08 06:00:57,126 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['default', 1, 0]) securityworker stdout | 2023-08-08 06:00:57,128 [77] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2', 32, True, True, 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4']) securityworker stdout | 2023-08-08 06:00:57,133 [77] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens securityworker stdout | 2023-08-08 06:00:57,134 [77] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 securityworker stdout | 2023-08-08 06:00:57,201 [77] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 securityworker stdout | 2023-08-08 06:00:57,203 [77] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["0i6EOn00SNe6M9Or3cPQcQ"], "expires_at": "2023-08-08T18:00:57.000000Z", "issued_at": "2023-08-08T06:00:57.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} securityworker stdout | 2023-08-08 06:00:57,204 [77] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 gunicorn-web stdout | 2023-08-08 06:00:57,266 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:00:57,304 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:00:57,313 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} securityworker stdout | 2023-08-08 06:00:57,313 [77] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "PUT /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.1" 201 0 securityworker stdout | 2023-08-08 06:00:57,313 [77] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 -X PUT -H "X-Auth-Token: gAAAAABk0doZI7J3..." securityworker stdout | 2023-08-08 06:00:57,313 [77] [DEBUG] [swiftclient] RESP STATUS: 201 Created securityworker stdout | 2023-08-08 06:00:57,313 [77] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Etag': '54a01009f17bdb7ec1dd1cb427244304', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:58 GMT', 'X-Trans-Id': 'tx27fb6e15a6da44fe9047f-0064d1da19', 'X-Openstack-Request-Id': 'tx27fb6e15a6da44fe9047f-0064d1da19', 'Date': 'Tue, 08 Aug 2023 06:00:57 GMT'} securityworker stdout | 2023-08-08 06:00:57,314 [77] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [4, 9]) securityworker stdout | 2023-08-08 06:00:57,319 [77] [DEBUG] [peewee] ('(SELECT * FROM (SELECT "t1"."id", "t1"."content_checksum", "t1"."image_size", "t1"."uuid", "t1"."cas_path", "t1"."uncompressed_size" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s) AS "q0") UNION ALL ((SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size" FROM "imagestorage" AS "t3" INNER JOIN "manifestblob" AS "t4" ON ("t4"."blob_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q1") UNION ALL (SELECT * FROM (SELECT "t5"."id", "t5"."content_checksum", "t5"."image_size", "t5"."uuid", "t5"."cas_path", "t5"."uncompressed_size" FROM "imagestorage" AS "t5" INNER JOIN "manifestblob" AS "t6" ON ("t6"."blob_id" = "t5"."id") WHERE (("t6"."repository_id" = %s) AND ("t5"."content_checksum" = %s)) LIMIT %s) AS "q2"))', [2, 'sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6', 1, 2, 'sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3', 1, 2, 'sha256:e6b52c3b1f61dec2ad1aa5e78bc737bff4d1f8709cb246b31adb518ca6146886', 1]) securityworker stdout | 2023-08-08 06:00:57,325 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,327 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,329 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,330 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) gunicorn-web stdout | 2023-08-08 06:00:57,331 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} securityworker stdout | 2023-08-08 06:00:57,332 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,333 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) gunicorn-web stdout | 2023-08-08 06:00:57,334 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} securityworker stdout | 2023-08-08 06:00:57,334 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,335 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,337 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,338 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,339 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,340 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,342 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,343 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,344 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,345 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,347 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,348 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,350 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,351 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,352 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,353 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,355 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['98053f66-67a2-4aae-b0f7-c22971a2984d']) securityworker stdout | 2023-08-08 06:00:57,356 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,358 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,359 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,361 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,362 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,364 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,365 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,366 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,368 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) securityworker stdout | 2023-08-08 06:00:57,369 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['2b34b77f-a78c-44ab-bea4-0e2a8e3e5e11']) securityworker stdout | 2023-08-08 06:00:57,370 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."manifest_id", "t1"."repository_id", "t1"."index_status", "t1"."error_json", "t1"."last_indexed", "t1"."indexer_hash", "t1"."indexer_version", "t1"."metadata_json" FROM "manifestsecuritystatus" AS "t1" WHERE ("t1"."manifest_id" = %s)', [1]) securityworker stdout | 2023-08-08 06:00:57,371 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) securityworker stdout | 2023-08-08 06:00:57,373 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) securityworker stdout | 2023-08-08 06:00:57,375 [77] [DEBUG] [data.secscan_model.secscan_v4_model] Indexing manifest [1] 2/github@sha256:e5b10fcf5c6656cf3f187efc16a331a868b4f07c3e0b07455ab80c5f4d1883b5 securityworker stdout | 2023-08-08 06:00:57,379 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) securityworker stdout | 2023-08-08 06:00:57,381 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) securityworker stdout | 2023-08-08 06:00:57,382 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:00:57,570 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:00:57,570 [77] [DEBUG] [util.secscan.v4.api] POSTing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_report gunicorn-registry stdout | 2023-08-08 06:00:58,028 [203] [DEBUG] [app] Starting request: urn:request:2ec46670-70af-4c98-8b4b-4a364ac6834f (/v2/build_test/github/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4) {'X-Forwarded-For': '10.130.2.1'} gunicorn-registry stdout | 2023-08-08 06:00:58,028 [203] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 'namespace_name': 'build_test', 'repo_name': 'github'} gunicorn-registry stdout | 2023-08-08 06:00:58,028 [205] [DEBUG] [app] Starting request: urn:request:98b9d4b4-4bed-4374-8d58-94062d60b9c1 (/v2/build_test/github/blobs/sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6) {'X-Forwarded-For': '10.130.2.1'} gunicorn-registry stdout | 2023-08-08 06:00:58,028 [209] [DEBUG] [app] Starting request: urn:request:39c26b7b-9b8d-4701-820a-269e62373cd1 (/v2/build_test/github/blobs/sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3) {'X-Forwarded-For': '10.130.2.1'} gunicorn-registry stdout | 2023-08-08 06:00:58,028 [203] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktcXVheS1lbnRlcnByaXNlLmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tIiwibmJmIjoxNjkxNDc0NDU3LCJpYXQiOjE2OTE0NzQ0NTcsImV4cCI6MTY5MTQ3NDUxNywic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImJ1aWxkX3Rlc3QvZ2l0aHViIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiY29udGV4dCI6eyJjb20uYXBvc3RpbGxlLnJvb3RzIjpudWxsLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQifX0.N8jWiheDVx6072YIWNw_e5YWYVtO36NhKDXJ8CARLKhRAhC7XsDaLL0PNm_dBz6mfoP8lT8A117qR7WT6Abe8qHCN-n3M3xPd0xOAkNtTIreJ2xJZm86-1pap6CT7QtB7JI7fTkgOf8jv7h_vRDx6CWAAUurGxYs94M8fjY-hk4MfTrXu2AUQe4WwhAW7MbxXE95OMg6yLznQTT0LzjEbg9Uq3lxeWsHfzTyyDplO5eqRlRl2_xbnedkUNpLJa9_QB5pvrrCaLgPiKX4hHqAgOLj0kcLYk-qAfIdyF3_zCnZjkf9BP4sq-XvjpvvnHnWBRgj1xzNt_Xif1Pj2w9D6g gunicorn-registry stdout | 2023-08-08 06:00:58,028 [203] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktcXVheS1lbnRlcnByaXNlLmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tIiwibmJmIjoxNjkxNDc0NDU3LCJpYXQiOjE2OTE0NzQ0NTcsImV4cCI6MTY5MTQ3NDUxNywic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImJ1aWxkX3Rlc3QvZ2l0aHViIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiY29udGV4dCI6eyJjb20uYXBvc3RpbGxlLnJvb3RzIjpudWxsLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQifX0.N8jWiheDVx6072YIWNw_e5YWYVtO36NhKDXJ8CARLKhRAhC7XsDaLL0PNm_dBz6mfoP8lT8A117qR7WT6Abe8qHCN-n3M3xPd0xOAkNtTIreJ2xJZm86-1pap6CT7QtB7JI7fTkgOf8jv7h_vRDx6CWAAUurGxYs94M8fjY-hk4MfTrXu2AUQe4WwhAW7MbxXE95OMg6yLznQTT0LzjEbg9Uq3lxeWsHfzTyyDplO5eqRlRl2_xbnedkUNpLJa9_QB5pvrrCaLgPiKX4hHqAgOLj0kcLYk-qAfIdyF3_zCnZjkf9BP4sq-XvjpvvnHnWBRgj1xzNt_Xif1Pj2w9D6g gunicorn-registry stdout | 2023-08-08 06:00:58,028 [209] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3', 'namespace_name': 'build_test', 'repo_name': 'github'} gunicorn-registry stdout | 2023-08-08 06:00:58,028 [205] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6', 'namespace_name': 'build_test', 'repo_name': 'github'} gunicorn-registry stdout | 2023-08-08 06:00:58,028 [209] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktcXVheS1lbnRlcnByaXNlLmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tIiwibmJmIjoxNjkxNDc0NDU3LCJpYXQiOjE2OTE0NzQ0NTcsImV4cCI6MTY5MTQ3NDUxNywic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImJ1aWxkX3Rlc3QvZ2l0aHViIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiY29udGV4dCI6eyJjb20uYXBvc3RpbGxlLnJvb3RzIjpudWxsLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQifX0.N8jWiheDVx6072YIWNw_e5YWYVtO36NhKDXJ8CARLKhRAhC7XsDaLL0PNm_dBz6mfoP8lT8A117qR7WT6Abe8qHCN-n3M3xPd0xOAkNtTIreJ2xJZm86-1pap6CT7QtB7JI7fTkgOf8jv7h_vRDx6CWAAUurGxYs94M8fjY-hk4MfTrXu2AUQe4WwhAW7MbxXE95OMg6yLznQTT0LzjEbg9Uq3lxeWsHfzTyyDplO5eqRlRl2_xbnedkUNpLJa9_QB5pvrrCaLgPiKX4hHqAgOLj0kcLYk-qAfIdyF3_zCnZjkf9BP4sq-XvjpvvnHnWBRgj1xzNt_Xif1Pj2w9D6g gunicorn-registry stdout | 2023-08-08 06:00:58,028 [205] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktcXVheS1lbnRlcnByaXNlLmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tIiwibmJmIjoxNjkxNDc0NDU3LCJpYXQiOjE2OTE0NzQ0NTcsImV4cCI6MTY5MTQ3NDUxNywic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImJ1aWxkX3Rlc3QvZ2l0aHViIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiY29udGV4dCI6eyJjb20uYXBvc3RpbGxlLnJvb3RzIjpudWxsLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQifX0.N8jWiheDVx6072YIWNw_e5YWYVtO36NhKDXJ8CARLKhRAhC7XsDaLL0PNm_dBz6mfoP8lT8A117qR7WT6Abe8qHCN-n3M3xPd0xOAkNtTIreJ2xJZm86-1pap6CT7QtB7JI7fTkgOf8jv7h_vRDx6CWAAUurGxYs94M8fjY-hk4MfTrXu2AUQe4WwhAW7MbxXE95OMg6yLznQTT0LzjEbg9Uq3lxeWsHfzTyyDplO5eqRlRl2_xbnedkUNpLJa9_QB5pvrrCaLgPiKX4hHqAgOLj0kcLYk-qAfIdyF3_zCnZjkf9BP4sq-XvjpvvnHnWBRgj1xzNt_Xif1Pj2w9D6g gunicorn-registry stdout | 2023-08-08 06:00:58,029 [209] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktcXVheS1lbnRlcnByaXNlLmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tIiwibmJmIjoxNjkxNDc0NDU3LCJpYXQiOjE2OTE0NzQ0NTcsImV4cCI6MTY5MTQ3NDUxNywic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImJ1aWxkX3Rlc3QvZ2l0aHViIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiY29udGV4dCI6eyJjb20uYXBvc3RpbGxlLnJvb3RzIjpudWxsLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQifX0.N8jWiheDVx6072YIWNw_e5YWYVtO36NhKDXJ8CARLKhRAhC7XsDaLL0PNm_dBz6mfoP8lT8A117qR7WT6Abe8qHCN-n3M3xPd0xOAkNtTIreJ2xJZm86-1pap6CT7QtB7JI7fTkgOf8jv7h_vRDx6CWAAUurGxYs94M8fjY-hk4MfTrXu2AUQe4WwhAW7MbxXE95OMg6yLznQTT0LzjEbg9Uq3lxeWsHfzTyyDplO5eqRlRl2_xbnedkUNpLJa9_QB5pvrrCaLgPiKX4hHqAgOLj0kcLYk-qAfIdyF3_zCnZjkf9BP4sq-XvjpvvnHnWBRgj1xzNt_Xif1Pj2w9D6g gunicorn-registry stdout | 2023-08-08 06:00:58,029 [205] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJwcG9Oa1llUXlmTFZac3hHQnBwUVVncThGUl9pR19nT04zbzVvcGNxakkifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheXJlZ2lzdHJ5LXF1YXktcXVheS1lbnRlcnByaXNlLmFwcHMud2h1NDEzb3NwMDMucWUuZGV2Y2x1c3Rlci5vcGVuc2hpZnQuY29tIiwibmJmIjoxNjkxNDc0NDU3LCJpYXQiOjE2OTE0NzQ0NTcsImV4cCI6MTY5MTQ3NDUxNywic3ViIjoiKGFub255bW91cykiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImJ1aWxkX3Rlc3QvZ2l0aHViIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiY29udGV4dCI6eyJjb20uYXBvc3RpbGxlLnJvb3RzIjpudWxsLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQifX0.N8jWiheDVx6072YIWNw_e5YWYVtO36NhKDXJ8CARLKhRAhC7XsDaLL0PNm_dBz6mfoP8lT8A117qR7WT6Abe8qHCN-n3M3xPd0xOAkNtTIreJ2xJZm86-1pap6CT7QtB7JI7fTkgOf8jv7h_vRDx6CWAAUurGxYs94M8fjY-hk4MfTrXu2AUQe4WwhAW7MbxXE95OMg6yLznQTT0LzjEbg9Uq3lxeWsHfzTyyDplO5eqRlRl2_xbnedkUNpLJa9_QB5pvrrCaLgPiKX4hHqAgOLj0kcLYk-qAfIdyF3_zCnZjkf9BP4sq-XvjpvvnHnWBRgj1xzNt_Xif1Pj2w9D6g gunicorn-registry stdout | 2023-08-08 06:00:58,031 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2023, 8, 8, 6, 0, 58, 29768), None, 'quay', 'quay', datetime.datetime(2023, 8, 8, 6, 0, 58, 29810), 'quay', None, datetime.datetime(2023, 8, 7, 6, 0, 58, 29832), datetime.datetime(2023, 8, 1, 6, 0, 58, 29849), None]) gunicorn-registry stdout | 2023-08-08 06:00:58,031 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2023, 8, 8, 6, 0, 58, 29896), None, 'quay', 'quay', datetime.datetime(2023, 8, 8, 6, 0, 58, 29941), 'quay', None, datetime.datetime(2023, 8, 7, 6, 0, 58, 29957), datetime.datetime(2023, 8, 1, 6, 0, 58, 29973), None]) gunicorn-registry stdout | 2023-08-08 06:00:58,031 [203] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2023-08-08 06:00:58,031 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2023, 8, 8, 6, 0, 58, 29946), None, 'quay', 'quay', datetime.datetime(2023, 8, 8, 6, 0, 58, 29980), 'quay', None, datetime.datetime(2023, 8, 7, 6, 0, 58, 29997), datetime.datetime(2023, 8, 1, 6, 0, 58, 30013), None]) gunicorn-registry stdout | 2023-08-08 06:00:58,031 [209] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2023-08-08 06:00:58,031 [205] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2023-08-08 06:00:58,036 [209] [DEBUG] [peewee.pool] Created new connection 140044981499520. gunicorn-registry stdout | 2023-08-08 06:00:58,036 [205] [DEBUG] [peewee.pool] Created new connection 140044981499520. gunicorn-registry stdout | 2023-08-08 06:00:58,037 [203] [DEBUG] [peewee.pool] Created new connection 140044981499520. gunicorn-registry stdout | 2023-08-08 06:00:58,043 [209] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2023-08-08 06:00:58,043 [209] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: (anonymous) gunicorn-registry stdout | 2023-08-08 06:00:58,043 [209] [DEBUG] [auth.registry_jwt_auth] Identity changed to (anonymous) gunicorn-registry stdout | 2023-08-08 06:00:58,043 [209] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'com.apostille.roots': None, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2023-08-08 06:00:58,043 [209] [DEBUG] [endpoints.v2] Checking permission for repo: build_test/github gunicorn-registry stdout | 2023-08-08 06:00:58,043 [203] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2023-08-08 06:00:58,043 [203] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: (anonymous) gunicorn-registry stdout | 2023-08-08 06:00:58,043 [205] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2023-08-08 06:00:58,043 [203] [DEBUG] [auth.registry_jwt_auth] Identity changed to (anonymous) gunicorn-registry stdout | 2023-08-08 06:00:58,043 [205] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: (anonymous) gunicorn-registry stdout | 2023-08-08 06:00:58,043 [205] [DEBUG] [auth.registry_jwt_auth] Identity changed to (anonymous) gunicorn-registry stdout | 2023-08-08 06:00:58,043 [209] [DEBUG] [data.cache.impl] Checking cache for key geo_restrictions__build_test gunicorn-registry stdout | 2023-08-08 06:00:58,043 [203] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'com.apostille.roots': None, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2023-08-08 06:00:58,043 [205] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'com.apostille.roots': None, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2023-08-08 06:00:58,043 [203] [DEBUG] [endpoints.v2] Checking permission for repo: build_test/github gunicorn-registry stdout | 2023-08-08 06:00:58,043 [205] [DEBUG] [endpoints.v2] Checking permission for repo: build_test/github gunicorn-registry stdout | 2023-08-08 06:00:58,043 [203] [DEBUG] [data.cache.impl] Checking cache for key geo_restrictions__build_test gunicorn-registry stdout | 2023-08-08 06:00:58,044 [205] [DEBUG] [data.cache.impl] Checking cache for key geo_restrictions__build_test gunicorn-registry stdout | 2023-08-08 06:00:58,044 [209] [DEBUG] [data.cache.impl] Found no result in cache for key geo_restrictions__build_test; calling loader gunicorn-registry stdout | 2023-08-08 06:00:58,044 [203] [DEBUG] [data.cache.impl] Found no result in cache for key geo_restrictions__build_test; calling loader gunicorn-registry stdout | 2023-08-08 06:00:58,044 [205] [DEBUG] [data.cache.impl] Found no result in cache for key geo_restrictions__build_test; calling loader gunicorn-registry stdout | 2023-08-08 06:00:58,044 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."added", "t1"."description", "t1"."unstructured_json", "t1"."restricted_region_iso_code" FROM "namespacegeorestriction" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-registry stdout | 2023-08-08 06:00:58,045 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."added", "t1"."description", "t1"."unstructured_json", "t1"."restricted_region_iso_code" FROM "namespacegeorestriction" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-registry stdout | 2023-08-08 06:00:58,045 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."added", "t1"."description", "t1"."unstructured_json", "t1"."restricted_region_iso_code" FROM "namespacegeorestriction" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-registry stdout | 2023-08-08 06:00:58,046 [209] [DEBUG] [data.cache.impl] Got loaded result for key geo_restrictions__build_test: [] gunicorn-registry stdout | 2023-08-08 06:00:58,046 [205] [DEBUG] [data.cache.impl] Got loaded result for key geo_restrictions__build_test: [] gunicorn-registry stdout | 2023-08-08 06:00:58,046 [209] [DEBUG] [data.cache.impl] Caching loaded result for key geo_restrictions__build_test with expiration []: 240s gunicorn-registry stdout | 2023-08-08 06:00:58,046 [203] [DEBUG] [data.cache.impl] Got loaded result for key geo_restrictions__build_test: [] gunicorn-registry stdout | 2023-08-08 06:00:58,046 [205] [DEBUG] [data.cache.impl] Caching loaded result for key geo_restrictions__build_test with expiration []: 240s gunicorn-registry stdout | 2023-08-08 06:00:58,046 [203] [DEBUG] [data.cache.impl] Caching loaded result for key geo_restrictions__build_test with expiration []: 240s gunicorn-registry stdout | 2023-08-08 06:00:58,046 [209] [DEBUG] [data.cache.impl] Cached loaded result for key geo_restrictions__build_test with expiration []: 240s gunicorn-registry stdout | 2023-08-08 06:00:58,047 [205] [DEBUG] [data.cache.impl] Cached loaded result for key geo_restrictions__build_test with expiration []: 240s gunicorn-registry stdout | 2023-08-08 06:00:58,047 [203] [DEBUG] [data.cache.impl] Cached loaded result for key geo_restrictions__build_test with expiration []: 240s gunicorn-registry stdout | 2023-08-08 06:00:58,047 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', True, 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,047 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', True, 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,047 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', True, 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,049 [203] [DEBUG] [endpoints.decorators] Skipping pull through proxy cache: instance matching query does not exist: gunicorn-registry stdout | SQL: SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s gunicorn-registry stdout | Params: ['build_test', True, 1, 0] gunicorn-registry stdout | 2023-08-08 06:00:58,049 [209] [DEBUG] [endpoints.decorators] Skipping pull through proxy cache: instance matching query does not exist: gunicorn-registry stdout | SQL: SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s gunicorn-registry stdout | Params: ['build_test', True, 1, 0] gunicorn-registry stdout | 2023-08-08 06:00:58,049 [203] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__build_test_github_sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4_2 gunicorn-registry stdout | 2023-08-08 06:00:58,049 [209] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__build_test_github_sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3_2 gunicorn-registry stdout | 2023-08-08 06:00:58,049 [203] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__build_test_github_sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4_2; calling loader gunicorn-registry stdout | 2023-08-08 06:00:58,049 [205] [DEBUG] [endpoints.decorators] Skipping pull through proxy cache: instance matching query does not exist: gunicorn-registry stdout | SQL: SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s gunicorn-registry stdout | Params: ['build_test', True, 1, 0] gunicorn-registry stdout | 2023-08-08 06:00:58,049 [209] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__build_test_github_sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3_2; calling loader gunicorn-registry stdout | 2023-08-08 06:00:58,049 [205] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__build_test_github_sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6_2 gunicorn-registry stdout | 2023-08-08 06:00:58,049 [205] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__build_test_github_sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6_2; calling loader gunicorn-registry stdout | 2023-08-08 06:00:58,050 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,050 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,051 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,053 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2023-08-08 06:00:58,053 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2023-08-08 06:00:58,053 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2023-08-08 06:00:58,054 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,054 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,055 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,055 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', ['sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,055 [209] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3', 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,056 [205] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6', 1, 0]) gunicorn-registry stdout | 2023-08-08 06:00:58,057 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2']) gunicorn-registry stdout | 2023-08-08 06:00:58,057 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['2b34b77f-a78c-44ab-bea4-0e2a8e3e5e11']) gunicorn-registry stdout | 2023-08-08 06:00:58,058 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['98053f66-67a2-4aae-b0f7-c22971a2984d']) gunicorn-registry stdout | 2023-08-08 06:00:58,058 [203] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2023-08-08 06:00:58,059 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2023-08-08 06:00:58,059 [203] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__build_test_github_sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4_2: {'uuid': 'a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2', 'digest': 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 'compressed_size': 32, 'uncompressed_size': None, 'uploading': True, 'db_id': 4, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'}} gunicorn-registry stdout | 2023-08-08 06:00:58,059 [203] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__build_test_github_sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4_2 with expiration {'uuid': 'a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2', 'digest': 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 'compressed_size': 32, 'uncompressed_size': None, 'uploading': True, 'db_id': 4, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'}}: 60s gunicorn-registry stdout | 2023-08-08 06:00:58,059 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2023-08-08 06:00:58,059 [203] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__build_test_github_sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4_2 with expiration {'uuid': 'a4ca7fa7-d3cd-470d-9c5e-5ad49b2d16b2', 'digest': 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 'compressed_size': 32, 'uncompressed_size': None, 'uploading': True, 'db_id': 4, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'}}: 60s gunicorn-registry stdout | 2023-08-08 06:00:58,059 [203] [DEBUG] [endpoints.v2.blob] Looking up the direct download URL for path: sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 gunicorn-registry stdout | 2023-08-08 06:00:58,059 [203] [DEBUG] [endpoints.v2.blob] Closing database connection before streaming layer data gunicorn-registry stdout | 2023-08-08 06:00:58,059 [203] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:00:58,059 [203] [DEBUG] [peewee.pool] Returning 140044981499520 to pool. gunicorn-registry stdout | 2023-08-08 06:00:58,060 [209] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['2b34b77f-a78c-44ab-bea4-0e2a8e3e5e11']) gunicorn-registry stdout | 2023-08-08 06:00:58,060 [203] [DEBUG] [app] Ending request: urn:request:2ec46670-70af-4c98-8b4b-4a364ac6834f (/v2/build_test/github/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4) {'endpoint': 'v2.download_blob', 'request_id': 'urn:request:2ec46670-70af-4c98-8b4b-4a364ac6834f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/v2/build_test/github/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 'path': '/v2/build_test/github/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'Go-http-client/2.0'} gunicorn-registry stdout | 2023-08-08 06:00:58,060 [205] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['98053f66-67a2-4aae-b0f7-c22971a2984d']) gunicorn-registry stdout | 2023-08-08 06:00:58,061 [209] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__build_test_github_sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3_2: {'uuid': '2b34b77f-a78c-44ab-bea4-0e2a8e3e5e11', 'digest': 'sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3', 'compressed_size': 20124293, 'uncompressed_size': 32184320, 'uploading': True, 'db_id': 1, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/1e/1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3'}} gunicorn-registry stdout | 2023-08-08 06:00:58,061 [209] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__build_test_github_sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3_2 with expiration {'uuid': '2b34b77f-a78c-44ab-bea4-0e2a8e3e5e11', 'digest': 'sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3', 'compressed_size': 20124293, 'uncompressed_size': 32184320, 'uploading': True, 'db_id': 1, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/1e/1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3'}}: 60s gunicorn-registry stdout | 2023-08-08 06:00:58,061 [205] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__build_test_github_sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6_2: {'uuid': '98053f66-67a2-4aae-b0f7-c22971a2984d', 'digest': 'sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6', 'compressed_size': 79896517, 'uncompressed_size': 217358848, 'uploading': True, 'db_id': 2, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/0d/0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6'}} gunicorn-registry stdout | 2023-08-08 06:00:58,061 [209] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__build_test_github_sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3_2 with expiration {'uuid': '2b34b77f-a78c-44ab-bea4-0e2a8e3e5e11', 'digest': 'sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3', 'compressed_size': 20124293, 'uncompressed_size': 32184320, 'uploading': True, 'db_id': 1, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/1e/1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3'}}: 60s gunicorn-registry stdout | 2023-08-08 06:00:58,061 [205] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__build_test_github_sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6_2 with expiration {'uuid': '98053f66-67a2-4aae-b0f7-c22971a2984d', 'digest': 'sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6', 'compressed_size': 79896517, 'uncompressed_size': 217358848, 'uploading': True, 'db_id': 2, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/0d/0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6'}}: 60s gunicorn-registry stdout | 2023-08-08 06:00:58,061 [209] [DEBUG] [endpoints.v2.blob] Looking up the direct download URL for path: sha256/1e/1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3 gunicorn-registry stdout | 2023-08-08 06:00:58,061 [205] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__build_test_github_sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6_2 with expiration {'uuid': '98053f66-67a2-4aae-b0f7-c22971a2984d', 'digest': 'sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6', 'compressed_size': 79896517, 'uncompressed_size': 217358848, 'uploading': True, 'db_id': 2, 'inputs': {'placements': ['default'], 'storage_path': 'sha256/0d/0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6'}}: 60s gunicorn-registry stdout | 2023-08-08 06:00:58,061 [203] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens gunicorn-registry stdout | 2023-08-08 06:00:58,061 [209] [DEBUG] [endpoints.v2.blob] Closing database connection before streaming layer data gunicorn-registry stdout | 2023-08-08 06:00:58,062 [209] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:00:58,062 [205] [DEBUG] [endpoints.v2.blob] Looking up the direct download URL for path: sha256/0d/0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6 gunicorn-registry stdout | 2023-08-08 06:00:58,062 [209] [DEBUG] [peewee.pool] Returning 140044981499520 to pool. gunicorn-registry stdout | 2023-08-08 06:00:58,062 [205] [DEBUG] [endpoints.v2.blob] Closing database connection before streaming layer data gunicorn-registry stdout | 2023-08-08 06:00:58,062 [205] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:00:58,062 [205] [DEBUG] [peewee.pool] Returning 140044981499520 to pool. gunicorn-registry stdout | 2023-08-08 06:00:58,062 [209] [DEBUG] [app] Ending request: urn:request:39c26b7b-9b8d-4701-820a-269e62373cd1 (/v2/build_test/github/blobs/sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3) {'endpoint': 'v2.download_blob', 'request_id': 'urn:request:39c26b7b-9b8d-4701-820a-269e62373cd1', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/v2/build_test/github/blobs/sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3', 'path': '/v2/build_test/github/blobs/sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'Go-http-client/2.0'} gunicorn-registry stdout | 2023-08-08 06:00:58,062 [205] [DEBUG] [app] Ending request: urn:request:98b9d4b4-4bed-4374-8d58-94062d60b9c1 (/v2/build_test/github/blobs/sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6) {'endpoint': 'v2.download_blob', 'request_id': 'urn:request:98b9d4b4-4bed-4374-8d58-94062d60b9c1', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/v2/build_test/github/blobs/sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6', 'path': '/v2/build_test/github/blobs/sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'Go-http-client/2.0'} gunicorn-registry stdout | 2023-08-08 06:00:58,064 [209] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens gunicorn-registry stdout | 2023-08-08 06:00:58,064 [205] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens gunicorn-registry stdout | 2023-08-08 06:00:58,065 [203] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 gunicorn-registry stdout | 2023-08-08 06:00:58,067 [209] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 gunicorn-registry stdout | 2023-08-08 06:00:58,067 [205] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 gunicorn-registry stdout | 2023-08-08 06:00:58,142 [209] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 gunicorn-registry stdout | 2023-08-08 06:00:58,145 [209] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["ed9DgRJ9SgiaZw7xPJROuQ"], "expires_at": "2023-08-08T18:00:58.000000Z", "issued_at": "2023-08-08T06:00:58.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} gunicorn-registry stdout | 2023-08-08 06:00:58,145 [205] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 gunicorn-registry stdout | 2023-08-08 06:00:58,147 [203] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 gunicorn-registry stdout | 2023-08-08 06:00:58,147 [209] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 gunicorn-registry stdout | 2023-08-08 06:00:58,149 [205] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["XDh4sYJEShO1CemgGraYWQ"], "expires_at": "2023-08-08T18:00:58.000000Z", "issued_at": "2023-08-08T06:00:58.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} gunicorn-registry stdout | 2023-08-08 06:00:58,150 [203] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["BcDwSXybTGyhXOrNGJLZvg"], "expires_at": "2023-08-08T18:00:58.000000Z", "issued_at": "2023-08-08T06:00:58.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} gunicorn-registry stdout | 2023-08-08 06:00:58,151 [205] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 gunicorn-registry stdout | 2023-08-08 06:00:58,153 [203] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 gunicorn-registry stdout | 2023-08-08 06:00:58,250 [203] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "GET /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.1" 200 32 gunicorn-registry stdout | 2023-08-08 06:00:58,251 [203] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 -X GET -H "X-Auth-Token: gAAAAABk0doa8kIl..." gunicorn-registry stdout | 2023-08-08 06:00:58,251 [203] [DEBUG] [swiftclient] RESP STATUS: 200 OK gunicorn-registry stdout | 2023-08-08 06:00:58,251 [203] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '54a01009f17bdb7ec1dd1cb427244304', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:58 GMT', 'X-Timestamp': '1691474457.27958', 'Accept-Ranges': 'bytes', 'Content-Length': '32', 'X-Trans-Id': 'txd0fbbb13714047de93666-0064d1da1a', 'X-Openstack-Request-Id': 'txd0fbbb13714047de93666-0064d1da1a', 'Date': 'Tue, 08 Aug 2023 06:00:58 GMT'} nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:00:58 +0000] "GET /v2/build_test/github/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/2.0" 200 32 "-" "Go-http-client/2.0" (0.225 878 0.224) gunicorn-registry stdout | 2023-08-08 06:00:58,251 [203] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:00:58 +0000] "GET /v2/build_test/github/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.1" 200 32 "-" "Go-http-client/2.0" gunicorn-registry stdout | 2023-08-08 06:00:58,264 [209] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "GET /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/1e/1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3 HTTP/1.1" 200 20124293 gunicorn-registry stdout | 2023-08-08 06:00:58,264 [209] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/1e/1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3 -X GET -H "X-Auth-Token: gAAAAABk0doa9n5s..." gunicorn-registry stdout | 2023-08-08 06:00:58,265 [209] [DEBUG] [swiftclient] RESP STATUS: 200 OK gunicorn-registry stdout | 2023-08-08 06:00:58,265 [209] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'X-Object-Manifest': 'quay/datastorage/registry/segments/da3bb2f6-298a-46c8-81b4-79b568997199', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:43 GMT', 'X-Timestamp': '1691474442.99799', 'Accept-Ranges': 'bytes', 'Content-Length': '20124293', 'Etag': '"f32aebe2eb967639e25f8b514fc33325"', 'X-Trans-Id': 'txed66c42c216c4f4fbb1e0-0064d1da1a', 'X-Openstack-Request-Id': 'txed66c42c216c4f4fbb1e0-0064d1da1a', 'Date': 'Tue, 08 Aug 2023 06:00:58 GMT'} gunicorn-registry stdout | 2023-08-08 06:00:58,270 [205] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "GET /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/0d/0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6 HTTP/1.1" 200 79896517 gunicorn-registry stdout | 2023-08-08 06:00:58,271 [205] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/sha256/0d/0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6 -X GET -H "X-Auth-Token: gAAAAABk0doanvrc..." gunicorn-registry stdout | 2023-08-08 06:00:58,271 [205] [DEBUG] [swiftclient] RESP STATUS: 200 OK gunicorn-registry stdout | 2023-08-08 06:00:58,271 [205] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'X-Object-Manifest': 'quay/datastorage/registry/segments/606362d9-ea09-4bd9-915d-b0b516b207e2', 'Last-Modified': 'Tue, 08 Aug 2023 03:32:17 GMT', 'X-Timestamp': '1691465536.98095', 'Accept-Ranges': 'bytes', 'Content-Length': '79896517', 'Etag': '"757d114cdb3eac26ad13014f3d4df276"', 'X-Trans-Id': 'tx1f91e14f6baa4997b320a-0064d1da1a', 'X-Openstack-Request-Id': 'tx1f91e14f6baa4997b320a-0064d1da1a', 'Date': 'Tue, 08 Aug 2023 06:00:58 GMT'} buildlogsarchiver stdout | 2023-08-08 06:00:58,317 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:00:58,489 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:00:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:00:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:09.417660+00:00 (in 9.999455 seconds) gcworker stdout | 2023-08-08 06:00:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:09 UTC)" (scheduled at 2023-08-08 06:00:59.417660+00:00) gcworker stdout | 2023-08-08 06:00:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264859418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:00:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:00:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:00:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:00:59,556 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gunicorn-registry stdout | 2023-08-08 06:00:59,922 [209] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:00:59 +0000] "GET /v2/build_test/github/blobs/sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3 HTTP/1.1" 200 20124293 "-" "Go-http-client/2.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:00:59 +0000] "GET /v2/build_test/github/blobs/sha256:1e228dd862f738ce04694b1270ea5a7afd77a1aa4a1779ec774f2d057695d2e3 HTTP/2.0" 200 19927325 "-" "Go-http-client/2.0" (1.896 79 1.896) exportactionlogsworker stdout | 2023-08-08 06:01:00,353 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:01:00,453 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:01:00,913 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 06:01:01,323 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:01:01,333 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:01:01,333 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:01:01,338 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:01:01,468 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:01:01,734 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:01:02,078 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:01:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:01:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:05.371682+00:00 (in 3.192634 seconds) notificationworker stdout | 2023-08-08 06:01:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:02 UTC)" (scheduled at 2023-08-08 06:01:02.178530+00:00) notificationworker stdout | 2023-08-08 06:01:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:01:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:01:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:01:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:12.293757+00:00 (in 9.999459 seconds) chunkcleanupworker stdout | 2023-08-08 06:01:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:12 UTC)" (scheduled at 2023-08-08 06:01:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:01:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:01:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 2, 294811), True, datetime.datetime(2023, 8, 8, 6, 1, 2, 294811), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:01:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:01:02,414 [205] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:01:02 +0000] "GET /v2/build_test/github/blobs/sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6 HTTP/1.1" 200 79896517 "-" "Go-http-client/2.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:01:02 +0000] "GET /v2/build_test/github/blobs/sha256:0d35fbf9cb3816d883b37290ca33531a982f7e7578db532791574ebc5f9122c6 HTTP/2.0" 200 78951764 "-" "Go-http-client/2.0" (4.389 80 4.388) gunicorn-registry stdout | 2023-08-08 06:01:03,094 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:01:03,098 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:01:03,101 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:01:03,103 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:01:03,106 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:01:03,114 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:01:03,122 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:01:03,126 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:01:03,130 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:01:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:01:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:09.793658+00:00 (in 5.605710 seconds) repositorygcworker stdout | 2023-08-08 06:01:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:04 UTC)" (scheduled at 2023-08-08 06:01:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:01:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:01:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:04 UTC)" executed successfully securityworker stdout | 2023-08-08 06:01:04,215 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "POST /indexer/api/v1/index_report HTTP/1.1" 201 None securityworker stdout | 2023-08-08 06:01:04,247 [77] [DEBUG] [peewee] ('DELETE FROM "manifestsecuritystatus" WHERE ("manifestsecuritystatus"."manifest_id" = %s)', [1]) securityworker stdout | 2023-08-08 06:01:04,249 [77] [DEBUG] [peewee] ('INSERT INTO "manifestsecuritystatus" ("manifest_id", "repository_id", "index_status", "error_json", "last_indexed", "indexer_hash", "indexer_version", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "manifestsecuritystatus"."id"', [1, 2, 2, '""', datetime.datetime(2023, 8, 8, 6, 1, 4, 248708), '57d6b7657adde342d823a8c046d051c0', 4, '{}']) securityworker stdout | 2023-08-08 06:01:04,255 [77] [INFO] [util.migrate.allocator] Marking id range as completed: 1-2 securityworker stdout | 2023-08-08 06:01:04,255 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:04,256 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 55, 56, 896654), 1, 2]) securityworker stdout | 2023-08-08 06:01:04,257 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:04,257 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:04,258 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 55, 56, 896654), 1, 2]) securityworker stdout | 2023-08-08 06:01:04,259 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:04,259 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:04,260 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:04,260 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:01:04,260 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:01:04,260 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:01:04,260 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:01:04,260 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:04,260 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:01:04,260 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:26 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:01:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:01:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:15.371682+00:00 (in 9.999474 seconds) notificationworker stdout | 2023-08-08 06:01:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:15 UTC)" (scheduled at 2023-08-08 06:01:05.371682+00:00) notificationworker stdout | 2023-08-08 06:01:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:01:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 5, 372782), True, datetime.datetime(2023, 8, 8, 6, 1, 5, 372782), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:01:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:01:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:01:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:01:07,179 [197] [DEBUG] [app] Starting request: urn:request:e5a4c991-6b0c-4640-918c-74431ea21796 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:01:07,180 [200] [DEBUG] [app] Starting request: urn:request:e1903d98-b995-41e9-8255-b5e7caee1a16 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:01:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:01:07,184 [207] [DEBUG] [app] Starting request: urn:request:5e7f07a0-231b-4a1c-a85a-1574a03ad0c2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:01:07,185 [207] [DEBUG] [app] Ending request: urn:request:5e7f07a0-231b-4a1c-a85a-1574a03ad0c2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5e7f07a0-231b-4a1c-a85a-1574a03ad0c2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:01:07,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:01:07,185 [210] [DEBUG] [app] Starting request: urn:request:db39ab4f-aab7-435b-b38d-056288f65faa (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:01:07,186 [210] [DEBUG] [app] Ending request: urn:request:db39ab4f-aab7-435b-b38d-056288f65faa (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:db39ab4f-aab7-435b-b38d-056288f65faa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:01:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:07,190 [200] [DEBUG] [app] Starting request: urn:request:480425d3-6a85-4aa0-9397-e5102446a407 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:07,190 [200] [DEBUG] [app] Ending request: urn:request:480425d3-6a85-4aa0-9397-e5102446a407 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:480425d3-6a85-4aa0-9397-e5102446a407', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:01:07,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:07,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:07,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:01:07,192 [198] [DEBUG] [app] Starting request: urn:request:0dca8d25-41ab-4f64-82a1-8dd16fb70167 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:07,193 [198] [DEBUG] [app] Ending request: urn:request:0dca8d25-41ab-4f64-82a1-8dd16fb70167 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0dca8d25-41ab-4f64-82a1-8dd16fb70167', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:01:07,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:01:07,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:01:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:01:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:37.197412+00:00 (in 29.999507 seconds) buildlogsarchiver stdout | 2023-08-08 06:01:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:37 UTC)" (scheduled at 2023-08-08 06:01:07.197412+00:00) gunicorn-web stdout | 2023-08-08 06:01:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:01:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 1, 7, 198319), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:01:07,202 [197] [DEBUG] [app] Ending request: urn:request:e5a4c991-6b0c-4640-918c-74431ea21796 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e5a4c991-6b0c-4640-918c-74431ea21796', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:01:07,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:07,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:01:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:01:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:01:07,203 [200] [DEBUG] [app] Ending request: urn:request:e1903d98-b995-41e9-8255-b5e7caee1a16 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e1903d98-b995-41e9-8255-b5e7caee1a16', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:01:07,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:01:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:01:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:01:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:01:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:01:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stderr | /quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py:52: FutureWarning: GzipFile was opened for writing, but this will change in future Python releases. Specify the mode argument for opening it for writing. buildlogsarchiver stderr | with GzipFile("testarchive", fileobj=tempfile) as zipstream: buildlogsarchiver stdout | 2023-08-08 06:01:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:01:07,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:01:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:01:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:19.417660+00:00 (in 9.999379 seconds) gcworker stdout | 2023-08-08 06:01:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:19 UTC)" (scheduled at 2023-08-08 06:01:09.417660+00:00) gcworker stdout | 2023-08-08 06:01:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264869418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:01:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:01:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:01:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:01:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:01:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:04.187429+00:00 (in 54.393253 seconds) repositorygcworker stdout | 2023-08-08 06:01:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:09 UTC)" (scheduled at 2023-08-08 06:01:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:01:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:01:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 9, 794669), True, datetime.datetime(2023, 8, 8, 6, 1, 9, 794669), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:01:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:01:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:01:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:09 UTC)" executed successfully builder stdout | 2023-08-08 06:01:11,348 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:01:11,357 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:01:11,357 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:01:11,362 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:01:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:22.293757+00:00 (in 9.999446 seconds) chunkcleanupworker stdout | 2023-08-08 06:01:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:22 UTC)" (scheduled at 2023-08-08 06:01:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:01:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:01:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 12, 294754), True, datetime.datetime(2023, 8, 8, 6, 1, 12, 294754), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:01:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:01:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:01:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:31.283850+00:00 (in 18.185931 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:01:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:13 UTC)" (scheduled at 2023-08-08 06:01:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:01:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:01:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 13, 98440), True, datetime.datetime(2023, 8, 8, 6, 1, 13, 98440), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:01:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:01:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:01:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:01:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:01:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:25.371682+00:00 (in 9.999490 seconds) notificationworker stdout | 2023-08-08 06:01:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:25 UTC)" (scheduled at 2023-08-08 06:01:15.371682+00:00) notificationworker stdout | 2023-08-08 06:01:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:01:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 15, 372670), True, datetime.datetime(2023, 8, 8, 6, 1, 15, 372670), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:01:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:01:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:01:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:01:17,508 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:01:18,755 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:01:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:01:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:29.417660+00:00 (in 9.999472 seconds) gcworker stdout | 2023-08-08 06:01:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:29 UTC)" (scheduled at 2023-08-08 06:01:19.417660+00:00) gcworker stdout | 2023-08-08 06:01:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264879418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:01:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:01:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:01:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:01:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:01:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:26.890741+00:00 (in 7.202855 seconds) securityworker stdout | 2023-08-08 06:01:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:49 UTC)" (scheduled at 2023-08-08 06:01:19.687363+00:00) securityworker stdout | 2023-08-08 06:01:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:01:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:01:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:01:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:01:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:01:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:01:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 56, 19, 691835), 1, 2]) securityworker stdout | 2023-08-08 06:01:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 56, 19, 691835), 1, 2]) securityworker stdout | 2023-08-08 06:01:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:01:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:01:19,905 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:01:20,348 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:01:21,372 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:01:21,381 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:01:21,381 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:01:21,386 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:22,179 [198] [DEBUG] [app] Starting request: urn:request:54df9b1c-7e06-4961-b9a8-816caec3998b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:01:22,180 [200] [DEBUG] [app] Starting request: urn:request:6b2460ec-e5e1-4bde-b68b-39c048218ace (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:01:22,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:22,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:22,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:01:22,185 [203] [DEBUG] [app] Starting request: urn:request:82e20bbc-58b7-4b53-831f-38160b18a7e5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:01:22,185 [203] [DEBUG] [app] Ending request: urn:request:82e20bbc-58b7-4b53-831f-38160b18a7e5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:82e20bbc-58b7-4b53-831f-38160b18a7e5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:01:22,185 [208] [DEBUG] [app] Starting request: urn:request:07dcce8a-7ed0-461d-8aa8-3515b50f1246 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:01:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:01:22,186 [208] [DEBUG] [app] Ending request: urn:request:07dcce8a-7ed0-461d-8aa8-3515b50f1246 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:07dcce8a-7ed0-461d-8aa8-3515b50f1246', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:01:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:01:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:22,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:22,191 [198] [DEBUG] [app] Starting request: urn:request:b2511dcc-3342-411c-922d-012b92f2a53b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:22,191 [197] [DEBUG] [app] Starting request: urn:request:e04cd33b-a55d-4983-b9ac-da9befe0a026 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:22,191 [198] [DEBUG] [app] Ending request: urn:request:b2511dcc-3342-411c-922d-012b92f2a53b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b2511dcc-3342-411c-922d-012b92f2a53b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:01:22,191 [197] [DEBUG] [app] Ending request: urn:request:e04cd33b-a55d-4983-b9ac-da9befe0a026 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e04cd33b-a55d-4983-b9ac-da9befe0a026', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:01:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:01:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:22,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:01:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:01:22,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:01:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:01:22,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:01:22,203 [198] [DEBUG] [app] Ending request: urn:request:54df9b1c-7e06-4961-b9a8-816caec3998b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:54df9b1c-7e06-4961-b9a8-816caec3998b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:01:22,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:22,203 [200] [DEBUG] [app] Ending request: urn:request:6b2460ec-e5e1-4bde-b68b-39c048218ace (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6b2460ec-e5e1-4bde-b68b-39c048218ace', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:01:22,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:01:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:01:22,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:01:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:01:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:01:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:01:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:01:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:01:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:28.874906+00:00 (in 6.580599 seconds) chunkcleanupworker stdout | 2023-08-08 06:01:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:32 UTC)" (scheduled at 2023-08-08 06:01:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:01:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:01:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 22, 294792), True, datetime.datetime(2023, 8, 8, 6, 1, 22, 294792), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:22,302 [57] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2023, 8, 8, 6, 6, 22, 294792), 1, '8d31c83e-003c-455b-a533-869e8f69dfb4', 6, 'b3bbe8e7-208c-45c4-8b4b-361fe4b5d3ea']) chunkcleanupworker stdout | 2023-08-08 06:01:22,306 [57] [DEBUG] [workers.queueworker] Queue gave us some work: {"location": "default", "uuid": "ba8e0298-0ea7-43ba-ac70-54becac72bf1", "path": "segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001"} chunkcleanupworker stdout | 2023-08-08 06:01:22,306 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:22,306 [57] [DEBUG] [__main__] Got chunk cleanup queue item: {'location': 'default', 'uuid': 'ba8e0298-0ea7-43ba-ac70-54becac72bf1', 'path': 'segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001'} chunkcleanupworker stdout | 2023-08-08 06:01:22,307 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:22,309 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:22,399 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:22,401 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["zOAC4I4QSIWku6UyoHSfKQ"], "expires_at": "2023-08-08T18:01:22.000000Z", "issued_at": "2023-08-08T06:01:22.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:22,403 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:22,513 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "HEAD /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 HTTP/1.1" 200 0 chunkcleanupworker stdout | 2023-08-08 06:01:22,513 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 -I -H "X-Auth-Token: gAAAAABk0doym4T0..." chunkcleanupworker stdout | 2023-08-08 06:01:22,513 [57] [DEBUG] [swiftclient] RESP STATUS: 200 OK chunkcleanupworker stdout | 2023-08-08 06:01:22,513 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:52 GMT', 'X-Timestamp': '1691474451.60876', 'Accept-Ranges': 'bytes', 'Content-Length': '0', 'X-Trans-Id': 'tx4c7c7f767f8941a4bd511-0064d1da32', 'X-Openstack-Request-Id': 'tx4c7c7f767f8941a4bd511-0064d1da32', 'Date': 'Tue, 08 Aug 2023 06:01:22 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:22,514 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:22,516 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:22,577 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:22,579 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["jpU2tNcqQTK2umYj4LnT7w"], "expires_at": "2023-08-08T18:01:22.000000Z", "issued_at": "2023-08-08T06:01:22.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:22,581 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:22,659 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "HEAD /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 HTTP/1.1" 200 0 chunkcleanupworker stdout | 2023-08-08 06:01:22,659 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 -I -H "X-Auth-Token: gAAAAABk0doyPgCY..." chunkcleanupworker stdout | 2023-08-08 06:01:22,659 [57] [DEBUG] [swiftclient] RESP STATUS: 200 OK chunkcleanupworker stdout | 2023-08-08 06:01:22,660 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:52 GMT', 'X-Timestamp': '1691474451.60876', 'Accept-Ranges': 'bytes', 'Content-Length': '0', 'X-Trans-Id': 'tx35f7a2a531bd4e4b879ac-0064d1da32', 'X-Openstack-Request-Id': 'tx35f7a2a531bd4e4b879ac-0064d1da32', 'Date': 'Tue, 08 Aug 2023 06:01:22 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:22,660 [57] [DEBUG] [storage.swift] Found headers for path segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 to delete: {'content-type': 'application/octet-stream', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'last-modified': 'Tue, 08 Aug 2023 06:00:52 GMT', 'x-timestamp': '1691474451.60876', 'accept-ranges': 'bytes', 'content-length': '0', 'x-trans-id': 'tx35f7a2a531bd4e4b879ac-0064d1da32', 'x-openstack-request-id': 'tx35f7a2a531bd4e4b879ac-0064d1da32', 'date': 'Tue, 08 Aug 2023 06:01:22 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:22,660 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:22,662 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:22,729 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:22,731 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["plo3ZwKBQSiPgWEPS3gAyg"], "expires_at": "2023-08-08T18:01:22.000000Z", "issued_at": "2023-08-08T06:01:22.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:22,732 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:22,841 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "DELETE /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 HTTP/1.1" 204 0 chunkcleanupworker stdout | 2023-08-08 06:01:22,841 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 DELETE -H "X-Auth-Token: gAAAAABk0doy9S_0..." chunkcleanupworker stdout | 2023-08-08 06:01:22,841 [57] [DEBUG] [swiftclient] RESP STATUS: 204 No Content chunkcleanupworker stdout | 2023-08-08 06:01:22,842 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx9a98d8f2d1ff44cdbf483-0064d1da32', 'X-Openstack-Request-Id': 'tx9a98d8f2d1ff44cdbf483-0064d1da32', 'Date': 'Tue, 08 Aug 2023 06:01:22 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:22,842 [57] [DEBUG] [peewee] ('DELETE FROM "queueitem" WHERE ("queueitem"."id" = %s)', [6]) chunkcleanupworker stdout | 2023-08-08 06:01:22,851 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 22, 850686), True, datetime.datetime(2023, 8, 8, 6, 1, 22, 850686), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:22,853 [57] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2023, 8, 8, 6, 6, 22, 850686), 1, 'c6101f59-29a7-4a0e-a91d-a82251339ca1', 5, '3578702d-ad7b-47f8-908c-348b45592c61']) chunkcleanupworker stdout | 2023-08-08 06:01:22,856 [57] [DEBUG] [workers.queueworker] Queue gave us some work: {"location": "default", "uuid": "ba8e0298-0ea7-43ba-ac70-54becac72bf1", "path": "segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000"} chunkcleanupworker stdout | 2023-08-08 06:01:22,856 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:22,856 [57] [DEBUG] [__main__] Got chunk cleanup queue item: {'location': 'default', 'uuid': 'ba8e0298-0ea7-43ba-ac70-54becac72bf1', 'path': 'segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000'} chunkcleanupworker stdout | 2023-08-08 06:01:22,857 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:22,858 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:22,915 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:22,916 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["u5jPPB61RS2W04_vqIMd6Q"], "expires_at": "2023-08-08T18:01:22.000000Z", "issued_at": "2023-08-08T06:01:22.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:22,918 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:22,999 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "HEAD /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000 HTTP/1.1" 200 0 chunkcleanupworker stdout | 2023-08-08 06:01:22,999 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000 -I -H "X-Auth-Token: gAAAAABk0doyE1v0..." chunkcleanupworker stdout | 2023-08-08 06:01:22,999 [57] [DEBUG] [swiftclient] RESP STATUS: 200 OK chunkcleanupworker stdout | 2023-08-08 06:01:22,999 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '7d9508378ab5f6c33b20297d21e21e09', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:51 GMT', 'X-Timestamp': '1691474450.77222', 'Accept-Ranges': 'bytes', 'Content-Length': '79896517', 'X-Trans-Id': 'txb6b0d11404fa4ef8836e7-0064d1da32', 'X-Openstack-Request-Id': 'txb6b0d11404fa4ef8836e7-0064d1da32', 'Date': 'Tue, 08 Aug 2023 06:01:22 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:23,000 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:23,001 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:23,054 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:23,056 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["WXUFJtIZSwS5oFl0JSKZCQ"], "expires_at": "2023-08-08T18:01:23.000000Z", "issued_at": "2023-08-08T06:01:23.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:23,057 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 gcworker stdout | 2023-08-08 06:01:23,077 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} chunkcleanupworker stdout | 2023-08-08 06:01:23,169 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "HEAD /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000 HTTP/1.1" 200 0 chunkcleanupworker stdout | 2023-08-08 06:01:23,169 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000 -I -H "X-Auth-Token: gAAAAABk0dozc5ck..." chunkcleanupworker stdout | 2023-08-08 06:01:23,169 [57] [DEBUG] [swiftclient] RESP STATUS: 200 OK chunkcleanupworker stdout | 2023-08-08 06:01:23,169 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': '7d9508378ab5f6c33b20297d21e21e09', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:51 GMT', 'X-Timestamp': '1691474450.77222', 'Accept-Ranges': 'bytes', 'Content-Length': '79896517', 'X-Trans-Id': 'txf28930683e604a068b1f9-0064d1da33', 'X-Openstack-Request-Id': 'txf28930683e604a068b1f9-0064d1da33', 'Date': 'Tue, 08 Aug 2023 06:01:23 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:23,169 [57] [DEBUG] [storage.swift] Found headers for path segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000 to delete: {'content-type': 'application/octet-stream', 'etag': '7d9508378ab5f6c33b20297d21e21e09', 'last-modified': 'Tue, 08 Aug 2023 06:00:51 GMT', 'x-timestamp': '1691474450.77222', 'accept-ranges': 'bytes', 'content-length': '79896517', 'x-trans-id': 'txf28930683e604a068b1f9-0064d1da33', 'x-openstack-request-id': 'txf28930683e604a068b1f9-0064d1da33', 'date': 'Tue, 08 Aug 2023 06:01:23 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:23,170 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:23,171 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:23,272 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:23,273 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["JtYghTMNSjOr5vyN3sa89Q"], "expires_at": "2023-08-08T18:01:23.000000Z", "issued_at": "2023-08-08T06:01:23.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:23,275 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:23,445 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "DELETE /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000 HTTP/1.1" 204 0 chunkcleanupworker stdout | 2023-08-08 06:01:23,446 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000000 DELETE -H "X-Auth-Token: gAAAAABk0dozyezo..." chunkcleanupworker stdout | 2023-08-08 06:01:23,446 [57] [DEBUG] [swiftclient] RESP STATUS: 204 No Content chunkcleanupworker stdout | 2023-08-08 06:01:23,446 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx0ebb94431dab4304aafda-0064d1da33', 'X-Openstack-Request-Id': 'tx0ebb94431dab4304aafda-0064d1da33', 'Date': 'Tue, 08 Aug 2023 06:01:23 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:23,446 [57] [DEBUG] [peewee] ('DELETE FROM "queueitem" WHERE ("queueitem"."id" = %s)', [5]) chunkcleanupworker stdout | 2023-08-08 06:01:23,540 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 23, 539297), True, datetime.datetime(2023, 8, 8, 6, 1, 23, 539297), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:23,542 [57] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2023, 8, 8, 6, 6, 23, 539297), 1, '6fcf31f4-63f5-4489-a246-ccaccb193c35', 8, 'abca9f80-e8f0-4412-83ed-7aa319042585']) chunkcleanupworker stdout | 2023-08-08 06:01:23,547 [57] [DEBUG] [workers.queueworker] Queue gave us some work: {"location": "default", "uuid": "3ddb5a3c-38ce-4a0d-a562-b832f9de990b", "path": "segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001"} chunkcleanupworker stdout | 2023-08-08 06:01:23,547 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:23,547 [57] [DEBUG] [__main__] Got chunk cleanup queue item: {'location': 'default', 'uuid': '3ddb5a3c-38ce-4a0d-a562-b832f9de990b', 'path': 'segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001'} chunkcleanupworker stdout | 2023-08-08 06:01:23,548 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:23,549 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:23,605 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:23,608 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["weJM5Oy6Qb-gCdoCuu0VpQ"], "expires_at": "2023-08-08T18:01:23.000000Z", "issued_at": "2023-08-08T06:01:23.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:23,610 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:23,686 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "HEAD /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 HTTP/1.1" 200 0 chunkcleanupworker stdout | 2023-08-08 06:01:23,686 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 -I -H "X-Auth-Token: gAAAAABk0dozapIU..." chunkcleanupworker stdout | 2023-08-08 06:01:23,686 [57] [DEBUG] [swiftclient] RESP STATUS: 200 OK chunkcleanupworker stdout | 2023-08-08 06:01:23,686 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:53 GMT', 'X-Timestamp': '1691474452.64566', 'Accept-Ranges': 'bytes', 'Content-Length': '0', 'X-Trans-Id': 'tx38f69e7accb64b1985461-0064d1da33', 'X-Openstack-Request-Id': 'tx38f69e7accb64b1985461-0064d1da33', 'Date': 'Tue, 08 Aug 2023 06:01:23 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:23,687 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:23,688 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:23,745 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:23,747 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["rPZtfyKZTrW7tk8AGzhnNw"], "expires_at": "2023-08-08T18:01:23.000000Z", "issued_at": "2023-08-08T06:01:23.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:23,749 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:23,848 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "HEAD /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 HTTP/1.1" 200 0 chunkcleanupworker stdout | 2023-08-08 06:01:23,849 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 -I -H "X-Auth-Token: gAAAAABk0doz8Buv..." chunkcleanupworker stdout | 2023-08-08 06:01:23,849 [57] [DEBUG] [swiftclient] RESP STATUS: 200 OK chunkcleanupworker stdout | 2023-08-08 06:01:23,849 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'application/octet-stream', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Last-Modified': 'Tue, 08 Aug 2023 06:00:53 GMT', 'X-Timestamp': '1691474452.64566', 'Accept-Ranges': 'bytes', 'Content-Length': '0', 'X-Trans-Id': 'tx02e1d768ea1149ffa32d7-0064d1da33', 'X-Openstack-Request-Id': 'tx02e1d768ea1149ffa32d7-0064d1da33', 'Date': 'Tue, 08 Aug 2023 06:01:23 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:23,849 [57] [DEBUG] [storage.swift] Found headers for path segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 to delete: {'content-type': 'application/octet-stream', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'last-modified': 'Tue, 08 Aug 2023 06:00:53 GMT', 'x-timestamp': '1691474452.64566', 'accept-ranges': 'bytes', 'content-length': '0', 'x-trans-id': 'tx02e1d768ea1149ffa32d7-0064d1da33', 'x-openstack-request-id': 'tx02e1d768ea1149ffa32d7-0064d1da33', 'date': 'Tue, 08 Aug 2023 06:01:23 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:23,849 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:23,854 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:23,909 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:23,911 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["0Nuan0X0RV6YDtKbzDgz2g"], "expires_at": "2023-08-08T18:01:23.000000Z", "issued_at": "2023-08-08T06:01:23.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:23,913 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:24,008 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "DELETE /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 HTTP/1.1" 204 0 chunkcleanupworker stdout | 2023-08-08 06:01:24,009 [57] [DEBUG] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 DELETE -H "X-Auth-Token: gAAAAABk0doz4EoK..." chunkcleanupworker stdout | 2023-08-08 06:01:24,009 [57] [DEBUG] [swiftclient] RESP STATUS: 204 No Content chunkcleanupworker stdout | 2023-08-08 06:01:24,009 [57] [DEBUG] [swiftclient] RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx25ad91c2f014455bb5fd5-0064d1da33', 'X-Openstack-Request-Id': 'tx25ad91c2f014455bb5fd5-0064d1da33', 'Date': 'Tue, 08 Aug 2023 06:01:24 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:24,009 [57] [DEBUG] [peewee] ('DELETE FROM "queueitem" WHERE ("queueitem"."id" = %s)', [8]) chunkcleanupworker stdout | 2023-08-08 06:01:24,019 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 24, 18511), True, datetime.datetime(2023, 8, 8, 6, 1, 24, 18511), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:24,021 [57] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2023, 8, 8, 6, 6, 24, 18511), 1, 'c9bfa4d3-efd8-40e3-8888-5241c4bb5143', 7, 'f3c26faa-db08-40c6-9b65-eaccef2b1301']) chunkcleanupworker stdout | 2023-08-08 06:01:24,024 [57] [DEBUG] [workers.queueworker] Queue gave us some work: {"location": "default", "uuid": "ba8e0298-0ea7-43ba-ac70-54becac72bf1", "path": "segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001"} chunkcleanupworker stdout | 2023-08-08 06:01:24,024 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:24,024 [57] [DEBUG] [__main__] Got chunk cleanup queue item: {'location': 'default', 'uuid': 'ba8e0298-0ea7-43ba-ac70-54becac72bf1', 'path': 'segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001'} chunkcleanupworker stdout | 2023-08-08 06:01:24,025 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:24,026 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:24,079 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:24,080 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["FJF6nd4KQbyiVOiSYchkLg"], "expires_at": "2023-08-08T18:01:24.000000Z", "issued_at": "2023-08-08T06:01:24.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:24,082 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:24,204 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "HEAD /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 HTTP/1.1" 404 0 chunkcleanupworker stdout | 2023-08-08 06:01:24,204 [57] [INFO] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/ba8e0298-0ea7-43ba-ac70-54becac72bf1/000000001 -I -H "X-Auth-Token: gAAAAABk0do0mSBf..." chunkcleanupworker stdout | 2023-08-08 06:01:24,204 [57] [INFO] [swiftclient] RESP STATUS: 404 Not Found chunkcleanupworker stdout | 2023-08-08 06:01:24,204 [57] [INFO] [swiftclient] RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx0c04b1f3bec3428c9938b-0064d1da34', 'X-Openstack-Request-Id': 'tx0c04b1f3bec3428c9938b-0064d1da34', 'Date': 'Tue, 08 Aug 2023 06:01:24 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:24,205 [57] [DEBUG] [__main__] Chunk already deleted chunkcleanupworker stdout | 2023-08-08 06:01:24,205 [57] [DEBUG] [peewee] ('DELETE FROM "queueitem" WHERE ("queueitem"."id" = %s)', [7]) chunkcleanupworker stdout | 2023-08-08 06:01:24,214 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 24, 213791), True, datetime.datetime(2023, 8, 8, 6, 1, 24, 213791), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:24,217 [57] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2023, 8, 8, 6, 6, 24, 213791), 1, '4e88f3ea-4069-41ab-9ffe-2c52f4125191', 9, 'b3c3bb6c-748f-4776-86a3-e5f35417032a']) chunkcleanupworker stdout | 2023-08-08 06:01:24,220 [57] [DEBUG] [workers.queueworker] Queue gave us some work: {"location": "default", "uuid": "3ddb5a3c-38ce-4a0d-a562-b832f9de990b", "path": "segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001"} chunkcleanupworker stdout | 2023-08-08 06:01:24,220 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:24,220 [57] [DEBUG] [__main__] Got chunk cleanup queue item: {'location': 'default', 'uuid': '3ddb5a3c-38ce-4a0d-a562-b832f9de990b', 'path': 'segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001'} chunkcleanupworker stdout | 2023-08-08 06:01:24,220 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] Making authentication request to https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens chunkcleanupworker stdout | 2023-08-08 06:01:24,222 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 chunkcleanupworker stdout | 2023-08-08 06:01:24,283 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000 "POST /v3/auth/tokens HTTP/1.1" 201 9119 chunkcleanupworker stdout | 2023-08-08 06:01:24,285 [57] [DEBUG] [keystoneclient.auth.identity.v3.base] {"token": {"methods": ["password"], "user": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1", "name": "psi-openshift-qe-jenkins", "password_expires_at": null}, "audit_ids": ["R0TOOZGsRQWgC1iznDKYxQ"], "expires_at": "2023-08-08T18:01:24.000000Z", "issued_at": "2023-08-08T06:01:24.000000Z", "project": {"domain": {"id": "62cf1b5ec006489db99e2b0ebfb55f57", "name": "redhat.com"}, "id": "542c6ebd48bf40fa857fc245c7572e30", "name": "openshift-qe-jenkins"}, "is_domain": false, "roles": [{"id": "7bc19f9c6c8a40b591e009c1487bc90b", "name": "swiftoperator"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "catalog": [{"endpoints": [{"id": "7e51926755004c999e955f5bf67f6a3e", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}, {"id": "8c4b14e0705045599addcfefd8c28e4c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13876", "region": "regionOne"}, {"id": "f0ab449b775a46e5b68a9a557e6b4fd3", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9876", "region": "regionOne"}], "id": "195742853db84a7a91cf9ee7c86b1527", "type": "load-balancer", "name": "octavia"}, {"endpoints": [{"id": "100e27a6663f4a668dfb4b54331f422a", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000", "region": "regionOne"}, {"id": "171878bb917f46cf8f0be10b967d2308", "interface": "admin", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:35357", "region": "regionOne"}, {"id": "db169bc8049d460aaa8153535385b795", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:5000", "region": "regionOne"}], "id": "1dd87b2d2f3c48f8b2efb6ec05114244", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "663d14235c7f4d669ed6a867d2cc44c4", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b0b0f2a6c48f40f9b02d5a24e055dddf", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "b626dda26c4644fbad561f518aea36d9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v3/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "52f9be8a2b474bb78da2ba2d8075da8d", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "42bb3d9b14e4485181391555e59a2108", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "54e23eafffb843fa9f96d2bf35bdd7d7", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "e3d44588ac794221a4b8cbfc77af1d89", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "57791433325c4534965fa3aa2d261efc", "type": "sharev2", "name": "manilav2"}, {"endpoints": [{"id": "5b3051eaadae4bb0a3b4782b7156fa3f", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "6430fefe62a94337970f7d31de27bb4d", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.4.1:8080", "region": "regionOne"}, {"id": "d203f3796d024fb3a24de86cad765d11", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.4.1:8080/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7965291ffa6541a6b994ac35090f65c1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0d3275bc1aeb4b34be24409bbf2edc78", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "0e73f32a92394c1db3d5eb263c3028d2", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a3c78f22c4e34299a542b366ed12de8a", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8786/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "7b6fd0b1b13c4686af2164703351e82f", "type": "share", "name": "manila"}, {"endpoints": [{"id": "1231edc7b0c248b981e41db15a98b1d9", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "5d97dfe3ffb841cbbdc333be5d1596eb", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "8c304efe816e42a7b5451fb974d0c62e", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13776/v2/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "92bbbbc605f84bac85570bf5f580debc", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "20d5c89370584fd49a58a779f9648476", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "a125c8f6cc7a4d8b93296928a97e6b43", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9292", "region": "regionOne"}, {"id": "b5b286d1a8784ee995d21f8c9d959b82", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13292", "region": "regionOne"}], "id": "a110cd3ee1c241818967b6a7686a6b0a", "type": "image", "name": "glance"}, {"endpoints": [{"id": "1f64c481a37d44708483f79eb1c629a1", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13005/v1", "region": "regionOne"}, {"id": "3019cee56463428e9af2bda9e0838d14", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}, {"id": "f92a0d175e354235a346e08d3a44fc74", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8000/v1", "region": "regionOne"}], "id": "a1e61f9ed9844aeb9e6f35ef22f8a805", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "7e4fa9f2a6894ed6813653b0cb421935", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "a9671ee6b35543f9827e2bb0d14d1952", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}, {"id": "c5326b5fcc054401a9998ad3f2ad264f", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8004/v1/542c6ebd48bf40fa857fc245c7572e30", "region": "regionOne"}], "id": "ac44d609d9784fc792271dedc36572c5", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "145063f862904cdf9840aaf2a9e4e3b9", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "37f42c9d47b4472ea78c1707c02d1d8e", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8778/placement", "region": "regionOne"}, {"id": "68aebfd675444d7e9642868e07863e99", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13778/placement", "region": "regionOne"}], "id": "cb194ff480f64df88a0adb01cbdfcbac", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "51b105226c4446aa856f595091517bed", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}, {"id": "9b462bda336a4a12aed30301763103b6", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13696", "region": "regionOne"}, {"id": "c75bf03f0b9c4afdba110db74669f5bd", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:9696", "region": "regionOne"}], "id": "cd676416d1924c25a0b25b0204d2dfad", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "74ab9f3494da417db04287f6ad6f672c", "interface": "public", "region_id": "regionOne", "url": "https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1", "region": "regionOne"}, {"id": "a83faf53b6ea40bdb0a84207cd29b524", "interface": "internal", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}, {"id": "a8f71ac09bed45c5a1a4c102163b8576", "interface": "admin", "region_id": "regionOne", "url": "http://172.16.21.59:8774/v2.1", "region": "regionOne"}], "id": "e263f59a0dc04f29a11e1e4cb53008f2", "type": "compute", "name": "nova"}, {"endpoints": [], "id": "f2a331bced1d4854acf8e2516a805c4b", "type": "volume", "name": "cinder"}]}} chunkcleanupworker stdout | 2023-08-08 06:01:24,287 [57] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 chunkcleanupworker stdout | 2023-08-08 06:01:24,508 [57] [DEBUG] [urllib3.connectionpool] https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808 "HEAD /v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 HTTP/1.1" 404 0 chunkcleanupworker stdout | 2023-08-08 06:01:24,509 [57] [INFO] [swiftclient] REQ: curl -i https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13808/v1/AUTH_542c6ebd48bf40fa857fc245c7572e30/quay/datastorage/registry/segments/3ddb5a3c-38ce-4a0d-a562-b832f9de990b/000000001 -I -H "X-Auth-Token: gAAAAABk0do0UBko..." chunkcleanupworker stdout | 2023-08-08 06:01:24,509 [57] [INFO] [swiftclient] RESP STATUS: 404 Not Found chunkcleanupworker stdout | 2023-08-08 06:01:24,509 [57] [INFO] [swiftclient] RESP HEADERS: {'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'X-Trans-Id': 'tx1d92c41436304241b4e2e-0064d1da34', 'X-Openstack-Request-Id': 'tx1d92c41436304241b4e2e-0064d1da34', 'Date': 'Tue, 08 Aug 2023 06:01:24 GMT'} chunkcleanupworker stdout | 2023-08-08 06:01:24,509 [57] [DEBUG] [__main__] Chunk already deleted chunkcleanupworker stdout | 2023-08-08 06:01:24,509 [57] [DEBUG] [peewee] ('DELETE FROM "queueitem" WHERE ("queueitem"."id" = %s)', [9]) chunkcleanupworker stdout | 2023-08-08 06:01:24,518 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 24, 517908), True, datetime.datetime(2023, 8, 8, 6, 1, 24, 517908), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:24,520 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:01:24,520 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:24,521 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:32 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:01:24,863 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:01:24,866 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:01:24,869 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:01:24,906 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:01:25,019 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:01:25,286 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 06:01:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:01:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:35.371682+00:00 (in 9.999484 seconds) notificationworker stdout | 2023-08-08 06:01:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:35 UTC)" (scheduled at 2023-08-08 06:01:25.371682+00:00) notificationworker stdout | 2023-08-08 06:01:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:01:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 25, 372686), True, datetime.datetime(2023, 8, 8, 6, 1, 25, 372686), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:01:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:01:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:01:25,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:01:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:01:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:47.398551+00:00 (in 21.883663 seconds) exportactionlogsworker stdout | 2023-08-08 06:01:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:25 UTC)" (scheduled at 2023-08-08 06:01:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:01:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:01:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:01:26,380 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:01:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:01:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:49.687363+00:00 (in 22.796066 seconds) securityworker stdout | 2023-08-08 06:01:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:56 UTC)" (scheduled at 2023-08-08 06:01:26.890741+00:00) securityworker stdout | 2023-08-08 06:01:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:01:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:01:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:01:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:01:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:01:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:01:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:01:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:01:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 56, 26, 896655), 1, 2]) securityworker stdout | 2023-08-08 06:01:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 56, 26, 896655), 1, 2]) securityworker stdout | 2023-08-08 06:01:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:01:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:01:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:01:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:01:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:01:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:01:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:01:26,937 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:01:27,046 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:01:27,274 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:01:27,315 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:01:27,329 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:01:27,344 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:01:27,348 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:01:28,341 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:01:28,506 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 06:01:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:01:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:32.293757+00:00 (in 3.418163 seconds) chunkcleanupworker stdout | 2023-08-08 06:01:28,876 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:28 UTC)" (scheduled at 2023-08-08 06:01:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:01:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:01:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:01:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:01:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:39.417660+00:00 (in 9.999441 seconds) gcworker stdout | 2023-08-08 06:01:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:39 UTC)" (scheduled at 2023-08-08 06:01:29.417660+00:00) gcworker stdout | 2023-08-08 06:01:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264889418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:01:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:01:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:01:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:01:29,594 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:01:30,373 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:01:30,474 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:01:30,924 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:01:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:01:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:54.284093+00:00 (in 22.999586 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:01:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:06:31 UTC)" (scheduled at 2023-08-08 06:01:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:01:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 1, 31, 284980), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:01:31,292 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 1, 31, 284980), True, datetime.datetime(2023, 8, 8, 6, 1, 31, 284980), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:01:31,294 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 1, 31, 284980), True, datetime.datetime(2023, 8, 8, 6, 1, 31, 284980), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 1, 31, 284980), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:01:31,295 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:01:31,295 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:06:31 UTC)" executed successfully builder stdout | 2023-08-08 06:01:31,396 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:01:31,404 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:01:31,405 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:01:31,409 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:01:31,477 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:01:31,773 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:01:32,116 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 06:01:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:01:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:42.293757+00:00 (in 9.999325 seconds) chunkcleanupworker stdout | 2023-08-08 06:01:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:42 UTC)" (scheduled at 2023-08-08 06:01:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:01:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:01:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 32, 294953), True, datetime.datetime(2023, 8, 8, 6, 1, 32, 294953), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:01:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:42 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:01:32,553 [197] [DEBUG] [app] Starting request: urn:request:bc5d5ca3-a590-41d3-a7b6-9248d3b1b263 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:01:32,553 [200] [DEBUG] [app] Starting request: urn:request:5c0316ee-483e-4e84-aaea-79268c8015c4 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:01:32,554 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:01:32,554 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:01:32,554 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:01:32,554 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:01:32,554 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:01:32,554 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:01:32,562 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:01:32,562 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:01:32,562 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:01:32,562 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:01:32,562 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:01:32,562 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:01:32,562 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:01:32,562 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:01:32,562 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:01:32,562 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:01:32,562 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:01:32,562 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:01:32,562 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:01:32,562 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:01:32,562 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:01:32,562 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:01:32,563 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 13, 32, 562958), datetime.datetime(2023, 8, 8, 6, 1, 32, 563012)]) gunicorn-web stdout | 2023-08-08 06:01:32,564 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:01:32,565 [197] [DEBUG] [app] Ending request: urn:request:bc5d5ca3-a590-41d3-a7b6-9248d3b1b263 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:bc5d5ca3-a590-41d3-a7b6-9248d3b1b263', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:01:32,565 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:32,566 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:01:32 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:01:32 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.014 62 0.013) gunicorn-web stdout | 2023-08-08 06:01:32,569 [200] [DEBUG] [app] Ending request: urn:request:5c0316ee-483e-4e84-aaea-79268c8015c4 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:5c0316ee-483e-4e84-aaea-79268c8015c4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:01:32,569 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:01:32 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 871 0.017) gunicorn-web stdout | 2023-08-08 06:01:32,570 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:01:32 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-registry stdout | 2023-08-08 06:01:33,106 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:01:33,109 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:01:33,112 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:01:33,117 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:01:33,119 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:01:33,123 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:01:33,130 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:01:33,138 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:01:33,142 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:01:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:01:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:45.371682+00:00 (in 9.999462 seconds) notificationworker stdout | 2023-08-08 06:01:35,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:45 UTC)" (scheduled at 2023-08-08 06:01:35.371682+00:00) notificationworker stdout | 2023-08-08 06:01:35,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:01:35,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 35, 373474), True, datetime.datetime(2023, 8, 8, 6, 1, 35, 373474), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:01:35,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:01:35,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:01:35,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:01:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:01:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:54.393627+00:00 (in 17.813199 seconds) namespacegcworker stdout | 2023-08-08 06:01:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:36 UTC)" (scheduled at 2023-08-08 06:01:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:01:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:01:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:01:37,178 [199] [DEBUG] [app] Starting request: urn:request:09b96991-079c-4927-8a7f-2f8005db77c3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:01:37,180 [200] [DEBUG] [app] Starting request: urn:request:592e2e33-957d-46eb-8dbb-aaff491d4f61 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:01:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:01:37,185 [203] [DEBUG] [app] Starting request: urn:request:16ebbd83-e1bd-4070-b3e2-0b9f372b2d82 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:01:37,185 [203] [DEBUG] [app] Ending request: urn:request:16ebbd83-e1bd-4070-b3e2-0b9f372b2d82 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:16ebbd83-e1bd-4070-b3e2-0b9f372b2d82', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:01:37,185 [209] [DEBUG] [app] Starting request: urn:request:48768b7c-e0a8-4ffe-a614-744b426fca78 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:01:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:37,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:01:37,186 [209] [DEBUG] [app] Ending request: urn:request:48768b7c-e0a8-4ffe-a614-744b426fca78 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:48768b7c-e0a8-4ffe-a614-744b426fca78', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:01:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:01:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:37,190 [200] [DEBUG] [app] Starting request: urn:request:6a07f5e3-9144-4c36-bc40-0555787aceaa (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:37,191 [200] [DEBUG] [app] Ending request: urn:request:6a07f5e3-9144-4c36-bc40-0555787aceaa (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6a07f5e3-9144-4c36-bc40-0555787aceaa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:01:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:37,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:37,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:01:37,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:37,193 [197] [DEBUG] [app] Starting request: urn:request:9a135cfa-bb37-4095-8a3e-edb027f631ac (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:37,193 [197] [DEBUG] [app] Ending request: urn:request:9a135cfa-bb37-4095-8a3e-edb027f631ac (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9a135cfa-bb37-4095-8a3e-edb027f631ac', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:01:37,194 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:37,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:37,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:01:37,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:01:37,198 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:01:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:07.197412+00:00 (in 29.999071 seconds) buildlogsarchiver stdout | 2023-08-08 06:01:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:07 UTC)" (scheduled at 2023-08-08 06:01:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:01:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 1, 37, 198788), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:01:37,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:01:37,203 [199] [DEBUG] [app] Ending request: urn:request:09b96991-079c-4927-8a7f-2f8005db77c3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:09b96991-079c-4927-8a7f-2f8005db77c3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:01:37,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:37,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:01:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:01:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:01:37,206 [200] [DEBUG] [app] Ending request: urn:request:592e2e33-957d-46eb-8dbb-aaff491d4f61 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:592e2e33-957d-46eb-8dbb-aaff491d4f61', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 06:01:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:01:37,206 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:37,207 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:01:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:01:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) buildlogsarchiver stdout | 2023-08-08 06:01:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:01:37,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:01:37,213 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:01:37,213 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:01:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:01:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:49.417660+00:00 (in 9.999461 seconds) gcworker stdout | 2023-08-08 06:01:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:49 UTC)" (scheduled at 2023-08-08 06:01:39.417660+00:00) gcworker stdout | 2023-08-08 06:01:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264899418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:01:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:01:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:01:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:49 UTC)" executed successfully builder stdout | 2023-08-08 06:01:41,420 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:01:41,429 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:01:41,430 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:01:41,435 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:01:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:52.293757+00:00 (in 9.999444 seconds) chunkcleanupworker stdout | 2023-08-08 06:01:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:52 UTC)" (scheduled at 2023-08-08 06:01:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:01:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:01:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 42, 294958), True, datetime.datetime(2023, 8, 8, 6, 1, 42, 294958), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:01:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:01:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:01:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:55.371682+00:00 (in 9.999483 seconds) notificationworker stdout | 2023-08-08 06:01:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:55 UTC)" (scheduled at 2023-08-08 06:01:45.371682+00:00) notificationworker stdout | 2023-08-08 06:01:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:01:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 45, 373249), True, datetime.datetime(2023, 8, 8, 6, 1, 45, 373249), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:01:45,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:01:45,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:01:45,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:01:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:01:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:25.514341+00:00 (in 38.115226 seconds) exportactionlogsworker stdout | 2023-08-08 06:01:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:47 UTC)" (scheduled at 2023-08-08 06:01:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:01:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:01:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 47, 399672), True, datetime.datetime(2023, 8, 8, 6, 1, 47, 399672), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:01:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:01:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:01:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:01:47,546 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:01:48,792 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:01:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:01:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:59.417660+00:00 (in 9.999384 seconds) gcworker stdout | 2023-08-08 06:01:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:59 UTC)" (scheduled at 2023-08-08 06:01:49.417660+00:00) gcworker stdout | 2023-08-08 06:01:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264909418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:01:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:01:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:01:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:01:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:01:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:01:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:01:56.890741+00:00 (in 7.202860 seconds) securityworker stdout | 2023-08-08 06:01:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:19 UTC)" (scheduled at 2023-08-08 06:01:49.687363+00:00) securityworker stdout | 2023-08-08 06:01:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:01:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:01:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:01:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:01:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:01:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:01:49,915 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:01:50,376 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:01:51,445 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:01:51,455 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:01:51,455 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:01:51,460 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:52,179 [200] [DEBUG] [app] Starting request: urn:request:03866b00-d481-4f72-9c42-99fc1c94f8bf (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:01:52,179 [198] [DEBUG] [app] Starting request: urn:request:b7e57728-1986-4094-949c-97d3286ff824 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:01:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:52,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:01:52,184 [206] [DEBUG] [app] Starting request: urn:request:f7c7475f-1d85-4363-ba98-56c2ef24e84f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:01:52,184 [209] [DEBUG] [app] Starting request: urn:request:73125856-6e47-407e-ad28-77a13aae995b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:01:52,185 [209] [DEBUG] [app] Ending request: urn:request:73125856-6e47-407e-ad28-77a13aae995b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:73125856-6e47-407e-ad28-77a13aae995b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:01:52,185 [206] [DEBUG] [app] Ending request: urn:request:f7c7475f-1d85-4363-ba98-56c2ef24e84f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f7c7475f-1d85-4363-ba98-56c2ef24e84f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:01:52,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:01:52,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:01:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:52,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:01:52,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:01:52,191 [197] [DEBUG] [app] Starting request: urn:request:eb5f080b-2b47-40ce-87d6-b8151c876a55 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:52,192 [199] [DEBUG] [app] Starting request: urn:request:6316a093-f065-4c05-b842-7aac625f6c62 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:01:52,192 [197] [DEBUG] [app] Ending request: urn:request:eb5f080b-2b47-40ce-87d6-b8151c876a55 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:eb5f080b-2b47-40ce-87d6-b8151c876a55', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:01:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:01:52,192 [199] [DEBUG] [app] Ending request: urn:request:6316a093-f065-4c05-b842-7aac625f6c62 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6316a093-f065-4c05-b842-7aac625f6c62', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:01:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:01:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:01:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:01:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:01:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:01:52,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:01:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:01:52,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:01:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:01:52,204 [198] [DEBUG] [app] Ending request: urn:request:b7e57728-1986-4094-949c-97d3286ff824 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b7e57728-1986-4094-949c-97d3286ff824', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:01:52,204 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:01:52,204 [200] [DEBUG] [app] Ending request: urn:request:03866b00-d481-4f72-9c42-99fc1c94f8bf (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:03866b00-d481-4f72-9c42-99fc1c94f8bf', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:01:52,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:01:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:01:52,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:01:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:01:52,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:01:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:01:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:01:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:01:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:02.293757+00:00 (in 9.999432 seconds) chunkcleanupworker stdout | 2023-08-08 06:01:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:02 UTC)" (scheduled at 2023-08-08 06:01:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:01:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:01:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 52, 294944), True, datetime.datetime(2023, 8, 8, 6, 1, 52, 294944), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:01:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:01:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:01:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:01:53,086 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:01:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:01:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:13.097375+00:00 (in 18.812783 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:01:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:54 UTC)" (scheduled at 2023-08-08 06:01:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:01:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:01:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:01:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:01:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:36.579841+00:00 (in 42.185667 seconds) namespacegcworker stdout | 2023-08-08 06:01:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:54 UTC)" (scheduled at 2023-08-08 06:01:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:01:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:01:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 54, 394645), True, datetime.datetime(2023, 8, 8, 6, 1, 54, 394645), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:01:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:01:54,404 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:01:54,404 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:02:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:01:54,871 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:01:54,874 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:01:54,877 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:01:54,935 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:01:55,050 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:01:55,315 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 06:01:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:01:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:02.178530+00:00 (in 6.806322 seconds) notificationworker stdout | 2023-08-08 06:01:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:05 UTC)" (scheduled at 2023-08-08 06:01:55.371682+00:00) notificationworker stdout | 2023-08-08 06:01:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:01:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 1, 55, 372709), True, datetime.datetime(2023, 8, 8, 6, 1, 55, 372709), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:01:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:01:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:01:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:01:56,402 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:01:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:01:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:19.687363+00:00 (in 22.796067 seconds) securityworker stdout | 2023-08-08 06:01:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:26 UTC)" (scheduled at 2023-08-08 06:01:56.890741+00:00) securityworker stdout | 2023-08-08 06:01:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:01:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:01:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:01:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:01:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:01:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:01:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:01:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:01:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 56, 56, 896336), 1, 2]) securityworker stdout | 2023-08-08 06:01:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 56, 56, 896336), 1, 2]) securityworker stdout | 2023-08-08 06:01:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:01:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:01:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:01:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:01:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:01:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:01:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:01:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:01:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:01:56,945 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:01:57,079 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:01:57,282 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:01:57,325 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:01:57,340 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:01:57,356 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:01:57,359 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:01:58,377 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:01:58,543 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:01:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:01:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:09.417660+00:00 (in 9.999478 seconds) gcworker stdout | 2023-08-08 06:01:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:09 UTC)" (scheduled at 2023-08-08 06:01:59.417660+00:00) gcworker stdout | 2023-08-08 06:01:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264919418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:01:59,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:01:59,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:01:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:01:59,613 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:02:00,408 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:02:00,511 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:02:00,932 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} builder stdout | 2023-08-08 06:02:01,470 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:02:01,479 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:02:01,479 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:02:01,484 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:02:01,492 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:02:01,790 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:02:02,153 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:02:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:02:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:05.371682+00:00 (in 3.192622 seconds) notificationworker stdout | 2023-08-08 06:02:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:02 UTC)" (scheduled at 2023-08-08 06:02:02.178530+00:00) notificationworker stdout | 2023-08-08 06:02:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:02:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:02:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:02:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:12.293757+00:00 (in 9.999360 seconds) chunkcleanupworker stdout | 2023-08-08 06:02:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:12 UTC)" (scheduled at 2023-08-08 06:02:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:02:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:02:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 2, 294917), True, datetime.datetime(2023, 8, 8, 6, 2, 2, 294917), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:02:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:02:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:02:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:02:03,120 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:02:03,124 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:02:03,127 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:02:03,130 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:02:03,133 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:02:03,136 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:02:03,142 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:02:03,149 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:02:03,151 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:02:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:02:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:09.793658+00:00 (in 5.605693 seconds) repositorygcworker stdout | 2023-08-08 06:02:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:04 UTC)" (scheduled at 2023-08-08 06:02:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:02:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:02:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:02:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:02:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:15.371682+00:00 (in 9.999487 seconds) notificationworker stdout | 2023-08-08 06:02:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:15 UTC)" (scheduled at 2023-08-08 06:02:05.371682+00:00) notificationworker stdout | 2023-08-08 06:02:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:02:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 5, 372693), True, datetime.datetime(2023, 8, 8, 6, 2, 5, 372693), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:02:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:02:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:02:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:02:07,178 [197] [DEBUG] [app] Starting request: urn:request:66893796-62ed-4ca3-9cdb-aca16d2c50dd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:02:07,179 [200] [DEBUG] [app] Starting request: urn:request:cadf2d9b-7a10-47db-9717-97212931d325 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:02:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:02:07,184 [204] [DEBUG] [app] Starting request: urn:request:c2daf67e-9afc-4d37-ae94-2f253a4eb785 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:02:07,184 [204] [DEBUG] [app] Ending request: urn:request:c2daf67e-9afc-4d37-ae94-2f253a4eb785 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c2daf67e-9afc-4d37-ae94-2f253a4eb785', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:02:07,184 [203] [DEBUG] [app] Starting request: urn:request:5c29eae9-3d7c-4518-adc1-73f90c3201b2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:02:07,185 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:02:07,185 [203] [DEBUG] [app] Ending request: urn:request:5c29eae9-3d7c-4518-adc1-73f90c3201b2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5c29eae9-3d7c-4518-adc1-73f90c3201b2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:02:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:02:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:02:07,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:07,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:07,190 [198] [DEBUG] [app] Starting request: urn:request:e2c99843-87ac-42e4-8012-31ee68b47c69 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:02:07,190 [198] [DEBUG] [app] Ending request: urn:request:e2c99843-87ac-42e4-8012-31ee68b47c69 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e2c99843-87ac-42e4-8012-31ee68b47c69', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:02:07,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:07,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:07,191 [199] [DEBUG] [app] Starting request: urn:request:e76361ab-5e43-4aca-a160-58fc03d880b6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:02:07,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:02:07,191 [199] [DEBUG] [app] Ending request: urn:request:e76361ab-5e43-4aca-a160-58fc03d880b6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e76361ab-5e43-4aca-a160-58fc03d880b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:02:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:02:07,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:02:07,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:02:07,198 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:02:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:37.197412+00:00 (in 29.999109 seconds) buildlogsarchiver stdout | 2023-08-08 06:02:07,199 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:37 UTC)" (scheduled at 2023-08-08 06:02:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:02:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 2, 7, 199177), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:02:07,201 [197] [DEBUG] [app] Ending request: urn:request:66893796-62ed-4ca3-9cdb-aca16d2c50dd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:66893796-62ed-4ca3-9cdb-aca16d2c50dd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:02:07,201 [200] [DEBUG] [app] Ending request: urn:request:cadf2d9b-7a10-47db-9717-97212931d325 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cadf2d9b-7a10-47db-9717-97212931d325', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:02:07,201 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:02:07,201 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:02:07,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:02:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:02:07,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:02:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:02:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:02:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.023) buildlogsarchiver stdout | 2023-08-08 06:02:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:02:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:02:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:02:07,212 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:02:07,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:02:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:02:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:19.417660+00:00 (in 9.999474 seconds) gcworker stdout | 2023-08-08 06:02:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:19 UTC)" (scheduled at 2023-08-08 06:02:09.417660+00:00) gcworker stdout | 2023-08-08 06:02:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264929418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:02:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:02:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:02:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:02:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:02:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:04.187429+00:00 (in 54.393218 seconds) repositorygcworker stdout | 2023-08-08 06:02:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:09 UTC)" (scheduled at 2023-08-08 06:02:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:02:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:02:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 9, 794719), True, datetime.datetime(2023, 8, 8, 6, 2, 9, 794719), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:02:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:02:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:02:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:09 UTC)" executed successfully builder stdout | 2023-08-08 06:02:11,494 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:02:11,504 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:02:11,504 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:02:11,509 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:02:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:02:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:22.293757+00:00 (in 9.999446 seconds) chunkcleanupworker stdout | 2023-08-08 06:02:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:22 UTC)" (scheduled at 2023-08-08 06:02:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:02:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:02:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 12, 294908), True, datetime.datetime(2023, 8, 8, 6, 2, 12, 294908), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:02:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:02:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:02:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:02:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:02:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:54.284093+00:00 (in 41.186130 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:02:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:13 UTC)" (scheduled at 2023-08-08 06:02:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:02:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:02:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 13, 98514), True, datetime.datetime(2023, 8, 8, 6, 2, 13, 98514), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:02:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:02:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:02:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:02:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:02:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:25.371682+00:00 (in 9.999481 seconds) notificationworker stdout | 2023-08-08 06:02:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:25 UTC)" (scheduled at 2023-08-08 06:02:15.371682+00:00) notificationworker stdout | 2023-08-08 06:02:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:02:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 15, 372682), True, datetime.datetime(2023, 8, 8, 6, 2, 15, 372682), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:02:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:02:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:02:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:02:17,572 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:02:18,800 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:02:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:02:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:29.417660+00:00 (in 9.999475 seconds) gcworker stdout | 2023-08-08 06:02:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:29 UTC)" (scheduled at 2023-08-08 06:02:19.417660+00:00) gcworker stdout | 2023-08-08 06:02:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264939418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:02:19,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:02:19,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:02:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:02:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:02:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:26.890741+00:00 (in 7.202864 seconds) securityworker stdout | 2023-08-08 06:02:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:49 UTC)" (scheduled at 2023-08-08 06:02:19.687363+00:00) securityworker stdout | 2023-08-08 06:02:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:02:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:02:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:02:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:02:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:02:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:02:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 57, 19, 691999), 1, 2]) securityworker stdout | 2023-08-08 06:02:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 57, 19, 691999), 1, 2]) securityworker stdout | 2023-08-08 06:02:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:02:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:02:19,927 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:02:20,384 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:02:21,519 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:02:21,528 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:02:21,528 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:02:21,533 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:02:22,178 [199] [DEBUG] [app] Starting request: urn:request:01973434-08df-408c-be6f-8d4ecc976b62 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:02:22,179 [197] [DEBUG] [app] Starting request: urn:request:9241b5b9-fc4f-44ad-8bf1-26c18007b46a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:02:22,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:02:22,184 [208] [DEBUG] [app] Starting request: urn:request:faedb48d-2971-4a1a-bd83-5fe549515b33 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:02:22,185 [208] [DEBUG] [app] Ending request: urn:request:faedb48d-2971-4a1a-bd83-5fe549515b33 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:faedb48d-2971-4a1a-bd83-5fe549515b33', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:02:22,185 [209] [DEBUG] [app] Starting request: urn:request:168dc22d-3eb4-4faf-9d19-c5f3b312d029 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:02:22,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:02:22,185 [209] [DEBUG] [app] Ending request: urn:request:168dc22d-3eb4-4faf-9d19-c5f3b312d029 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:168dc22d-3eb4-4faf-9d19-c5f3b312d029', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:02:22,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:02:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:22,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:22,191 [197] [DEBUG] [app] Starting request: urn:request:9a19c8a0-530b-4bb9-ad4b-6a0353b0e855 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:02:22,191 [197] [DEBUG] [app] Ending request: urn:request:9a19c8a0-530b-4bb9-ad4b-6a0353b0e855 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9a19c8a0-530b-4bb9-ad4b-6a0353b0e855', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:02:22,192 [200] [DEBUG] [app] Starting request: urn:request:7d823df9-47b1-442f-bd88-087abf734b9b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:02:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:22,192 [200] [DEBUG] [app] Ending request: urn:request:7d823df9-47b1-442f-bd88-087abf734b9b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7d823df9-47b1-442f-bd88-087abf734b9b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:02:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:02:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:22,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:02:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:02:22,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:02:22,203 [197] [DEBUG] [app] Ending request: urn:request:9241b5b9-fc4f-44ad-8bf1-26c18007b46a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9241b5b9-fc4f-44ad-8bf1-26c18007b46a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:02:22,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:02:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 06:02:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:02:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:02:22,204 [199] [DEBUG] [app] Ending request: urn:request:01973434-08df-408c-be6f-8d4ecc976b62 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:01973434-08df-408c-be6f-8d4ecc976b62', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:02:22,204 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:02:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:02:22,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:02:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:02:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:02:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:28.874906+00:00 (in 6.580622 seconds) chunkcleanupworker stdout | 2023-08-08 06:02:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:32 UTC)" (scheduled at 2023-08-08 06:02:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:02:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:02:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 22, 294779), True, datetime.datetime(2023, 8, 8, 6, 2, 22, 294779), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:02:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:02:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:02:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:02:23,108 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:02:24,882 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:02:24,885 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:02:24,888 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:02:24,971 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:02:25,083 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:02:25,352 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 06:02:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:02:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:35.371682+00:00 (in 9.999492 seconds) notificationworker stdout | 2023-08-08 06:02:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:35 UTC)" (scheduled at 2023-08-08 06:02:25.371682+00:00) notificationworker stdout | 2023-08-08 06:02:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:02:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 25, 372684), True, datetime.datetime(2023, 8, 8, 6, 2, 25, 372684), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:02:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:02:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:02:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:02:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:02:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:47.398551+00:00 (in 21.883599 seconds) exportactionlogsworker stdout | 2023-08-08 06:02:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:25 UTC)" (scheduled at 2023-08-08 06:02:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:02:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:02:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:02:26,421 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:02:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:02:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:49.687363+00:00 (in 22.796105 seconds) securityworker stdout | 2023-08-08 06:02:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:56 UTC)" (scheduled at 2023-08-08 06:02:26.890741+00:00) securityworker stdout | 2023-08-08 06:02:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:02:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:02:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:02:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:02:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:02:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:02:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:02:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:02:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:02:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:02:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 57, 26, 896573), 1, 2]) securityworker stdout | 2023-08-08 06:02:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 57, 26, 896573), 1, 2]) securityworker stdout | 2023-08-08 06:02:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:02:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:02:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:02:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:02:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:02:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:02:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:02:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:02:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:02:26,982 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:02:27,089 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:02:27,290 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:02:27,336 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:02:27,353 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:02:27,369 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:02:27,373 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:02:28,388 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:02:28,578 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 06:02:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:02:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:32.293757+00:00 (in 3.418312 seconds) chunkcleanupworker stdout | 2023-08-08 06:02:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:28 UTC)" (scheduled at 2023-08-08 06:02:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:02:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:02:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:02:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:02:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:39.417660+00:00 (in 9.999491 seconds) gcworker stdout | 2023-08-08 06:02:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:39 UTC)" (scheduled at 2023-08-08 06:02:29.417660+00:00) gcworker stdout | 2023-08-08 06:02:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264949418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:02:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:02:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:02:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:02:29,640 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:02:30,443 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:02:30,548 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:02:30,951 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:02:31,530 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} builder stdout | 2023-08-08 06:02:31,543 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:02:31,553 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:02:31,553 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:02:31,557 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:02:31,829 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:02:32,192 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 06:02:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:02:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:42.293757+00:00 (in 9.999450 seconds) chunkcleanupworker stdout | 2023-08-08 06:02:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:42 UTC)" (scheduled at 2023-08-08 06:02:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:02:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:02:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 32, 294802), True, datetime.datetime(2023, 8, 8, 6, 2, 32, 294802), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:02:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:02:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:02:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:02:33,133 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:02:33,136 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:02:33,139 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:02:33,141 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:02:33,144 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:02:33,147 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:02:33,156 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:02:33,160 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:02:33,163 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:02:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:02:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:45.371682+00:00 (in 9.999475 seconds) notificationworker stdout | 2023-08-08 06:02:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:45 UTC)" (scheduled at 2023-08-08 06:02:35.371682+00:00) notificationworker stdout | 2023-08-08 06:02:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:02:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 35, 372746), True, datetime.datetime(2023, 8, 8, 6, 2, 35, 372746), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:02:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:02:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:02:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:02:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:02:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:54.393627+00:00 (in 17.813277 seconds) namespacegcworker stdout | 2023-08-08 06:02:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:36 UTC)" (scheduled at 2023-08-08 06:02:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:02:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:02:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:02:37,179 [197] [DEBUG] [app] Starting request: urn:request:aea6c69a-b3d5-47af-9cbf-630e4d94e6dd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:02:37,180 [200] [DEBUG] [app] Starting request: urn:request:882436e9-feeb-4940-aa0a-a25e2c2aa57f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:02:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:37,183 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:37,186 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:02:37,186 [209] [DEBUG] [app] Starting request: urn:request:99e80b9f-55e0-4bfa-9832-43d1c1d142b4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:02:37,186 [209] [DEBUG] [app] Ending request: urn:request:99e80b9f-55e0-4bfa-9832-43d1c1d142b4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:99e80b9f-55e0-4bfa-9832-43d1c1d142b4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-registry stdout | 2023-08-08 06:02:37,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:02:37,187 [207] [DEBUG] [app] Starting request: urn:request:32e7f2d9-5cc8-4dd5-b4c3-e1ecd701ec8d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:02:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:02:37,188 [207] [DEBUG] [app] Ending request: urn:request:32e7f2d9-5cc8-4dd5-b4c3-e1ecd701ec8d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:32e7f2d9-5cc8-4dd5-b4c3-e1ecd701ec8d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:02:37,188 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:37,188 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:37,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:37,190 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:37,192 [200] [DEBUG] [app] Starting request: urn:request:1195c499-aaea-4bdd-98bc-4d62da446591 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:02:37,193 [200] [DEBUG] [app] Ending request: urn:request:1195c499-aaea-4bdd-98bc-4d62da446591 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1195c499-aaea-4bdd-98bc-4d62da446591', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:02:37,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:37,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:37,194 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:37,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:02:37,195 [198] [DEBUG] [app] Starting request: urn:request:4e60d535-f4e6-4583-b9f2-b96b1241ca8e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:02:37,195 [198] [DEBUG] [app] Ending request: urn:request:4e60d535-f4e6-4583-b9f2-b96b1241ca8e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4e60d535-f4e6-4583-b9f2-b96b1241ca8e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:02:37,195 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:37,196 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:37,196 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:02:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:02:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:07.197412+00:00 (in 29.999517 seconds) buildlogsarchiver stdout | 2023-08-08 06:02:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:07 UTC)" (scheduled at 2023-08-08 06:02:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:02:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 2, 37, 198393), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:02:37,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:02:37,201 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:02:37,205 [197] [DEBUG] [app] Ending request: urn:request:aea6c69a-b3d5-47af-9cbf-630e4d94e6dd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:aea6c69a-b3d5-47af-9cbf-630e4d94e6dd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:02:37,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:02:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:02:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:02:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:02:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:02:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 gunicorn-web stdout | 2023-08-08 06:02:37,207 [200] [DEBUG] [app] Ending request: urn:request:882436e9-feeb-4940-aa0a-a25e2c2aa57f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:882436e9-feeb-4940-aa0a-a25e2c2aa57f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:02:37,207 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:02:37,207 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:02:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:02:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:02:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:02:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:02:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:02:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:02:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:49.417660+00:00 (in 9.999502 seconds) gcworker stdout | 2023-08-08 06:02:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:49 UTC)" (scheduled at 2023-08-08 06:02:39.417660+00:00) gcworker stdout | 2023-08-08 06:02:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264959418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:02:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:02:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:02:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:49 UTC)" executed successfully builder stdout | 2023-08-08 06:02:41,568 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:02:41,578 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:02:41,578 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:02:41,583 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:02:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:02:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:52.293757+00:00 (in 9.999482 seconds) chunkcleanupworker stdout | 2023-08-08 06:02:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:52 UTC)" (scheduled at 2023-08-08 06:02:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:02:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:02:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 42, 294772), True, datetime.datetime(2023, 8, 8, 6, 2, 42, 294772), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:02:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:02:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:02:42,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:02:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:02:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:55.371682+00:00 (in 9.999443 seconds) notificationworker stdout | 2023-08-08 06:02:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:55 UTC)" (scheduled at 2023-08-08 06:02:45.371682+00:00) notificationworker stdout | 2023-08-08 06:02:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:02:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 45, 373405), True, datetime.datetime(2023, 8, 8, 6, 2, 45, 373405), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:02:45,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:02:45,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:02:45,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:02:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:02:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:25.514341+00:00 (in 38.115269 seconds) exportactionlogsworker stdout | 2023-08-08 06:02:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:47 UTC)" (scheduled at 2023-08-08 06:02:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:02:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:02:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 47, 399627), True, datetime.datetime(2023, 8, 8, 6, 2, 47, 399627), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:02:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:02:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:02:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:02:47,609 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:02:48,809 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:02:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:02:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:59.417660+00:00 (in 9.999476 seconds) gcworker stdout | 2023-08-08 06:02:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:59 UTC)" (scheduled at 2023-08-08 06:02:49.417660+00:00) gcworker stdout | 2023-08-08 06:02:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264969418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:02:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:02:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:02:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:02:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:02:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:02:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:02:56.890741+00:00 (in 7.202892 seconds) securityworker stdout | 2023-08-08 06:02:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:19 UTC)" (scheduled at 2023-08-08 06:02:49.687363+00:00) securityworker stdout | 2023-08-08 06:02:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:02:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:02:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:02:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:02:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:02:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:02:49,962 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:02:50,404 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:02:51,584 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:02:51,594 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:02:51,594 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:02:51,598 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:02:52,179 [198] [DEBUG] [app] Starting request: urn:request:5084909d-ebc4-4700-8e92-e5499de016fb (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:02:52,180 [197] [DEBUG] [app] Starting request: urn:request:8962567f-a5cc-4d5c-b73e-6c46264657e4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:02:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:52,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:02:52,184 [203] [DEBUG] [app] Starting request: urn:request:0d6873da-168c-460b-ad16-feebb2f715d9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:02:52,185 [210] [DEBUG] [app] Starting request: urn:request:c5daed24-39bd-44ad-84bd-702a0b4cb261 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:02:52,185 [203] [DEBUG] [app] Ending request: urn:request:0d6873da-168c-460b-ad16-feebb2f715d9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0d6873da-168c-460b-ad16-feebb2f715d9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:02:52,185 [210] [DEBUG] [app] Ending request: urn:request:c5daed24-39bd-44ad-84bd-702a0b4cb261 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c5daed24-39bd-44ad-84bd-702a0b4cb261', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:02:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:02:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-web stdout | 2023-08-08 06:02:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:52,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:02:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:52,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:02:52,191 [199] [DEBUG] [app] Starting request: urn:request:4f71551f-aecd-416a-9043-7ed4193fe06a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:02:52,192 [199] [DEBUG] [app] Ending request: urn:request:4f71551f-aecd-416a-9043-7ed4193fe06a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4f71551f-aecd-416a-9043-7ed4193fe06a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:02:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:02:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:52,192 [200] [DEBUG] [app] Starting request: urn:request:6b99d667-9dde-49b0-85e5-9c418e339868 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:02:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:02:52,193 [200] [DEBUG] [app] Ending request: urn:request:6b99d667-9dde-49b0-85e5-9c418e339868 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6b99d667-9dde-49b0-85e5-9c418e339868', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:02:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2023-08-08 06:02:52,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:02:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:02:52,194 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:02:52,194 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:02:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:02:52,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:02:52,203 [197] [DEBUG] [app] Ending request: urn:request:8962567f-a5cc-4d5c-b73e-6c46264657e4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8962567f-a5cc-4d5c-b73e-6c46264657e4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:02:52,204 [198] [DEBUG] [app] Ending request: urn:request:5084909d-ebc4-4700-8e92-e5499de016fb (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5084909d-ebc4-4700-8e92-e5499de016fb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:02:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:02:52,204 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:02:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:02:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:02:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:02:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:02:52,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:02:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:02:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:02:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:02.293757+00:00 (in 9.999456 seconds) chunkcleanupworker stdout | 2023-08-08 06:02:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:02 UTC)" (scheduled at 2023-08-08 06:02:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:02:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:02:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 52, 294788), True, datetime.datetime(2023, 8, 8, 6, 2, 52, 294788), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:02:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:02:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:02:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:02:53,138 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:02:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:02:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:13.097375+00:00 (in 18.812717 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:02:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:54 UTC)" (scheduled at 2023-08-08 06:02:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:02:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:02:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:02:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:02:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:36.579841+00:00 (in 42.185624 seconds) namespacegcworker stdout | 2023-08-08 06:02:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:54 UTC)" (scheduled at 2023-08-08 06:02:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:02:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:02:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 54, 394854), True, datetime.datetime(2023, 8, 8, 6, 2, 54, 394854), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:02:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:02:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:02:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:03:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:02:54,893 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:02:54,896 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:02:54,898 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:02:55,005 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:02:55,121 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:02:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:02:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:02.178530+00:00 (in 6.806292 seconds) notificationworker stdout | 2023-08-08 06:02:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:05 UTC)" (scheduled at 2023-08-08 06:02:55.371682+00:00) notificationworker stdout | 2023-08-08 06:02:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:02:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 2, 55, 372753), True, datetime.datetime(2023, 8, 8, 6, 2, 55, 372753), 0, 'notification/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:02:55,378 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} notificationworker stdout | 2023-08-08 06:02:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:02:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:02:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:05 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:02:56,457 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:02:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:02:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:19.687363+00:00 (in 22.796126 seconds) securityworker stdout | 2023-08-08 06:02:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:26 UTC)" (scheduled at 2023-08-08 06:02:56.890741+00:00) securityworker stdout | 2023-08-08 06:02:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:02:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:02:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:02:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:02:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:02:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:02:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:02:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:02:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:02:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:02:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 57, 56, 896463), 1, 2]) securityworker stdout | 2023-08-08 06:02:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 57, 56, 896463), 1, 2]) securityworker stdout | 2023-08-08 06:02:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:02:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:02:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:02:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:02:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:02:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:02:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:02:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:02:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:02:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:02:56,992 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:02:57,110 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:02:57,299 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:02:57,348 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:02:57,364 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:02:57,382 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:02:57,385 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:02:58,426 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:02:58,602 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:02:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:02:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:09.417660+00:00 (in 9.999475 seconds) gcworker stdout | 2023-08-08 06:02:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:09 UTC)" (scheduled at 2023-08-08 06:02:59.417660+00:00) gcworker stdout | 2023-08-08 06:02:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264979418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:02:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:02:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:02:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:02:59,648 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:03:00,480 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:03:00,577 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:03:00,987 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:03:01,558 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} builder stdout | 2023-08-08 06:03:01,608 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:03:01,618 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:03:01,618 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:03:01,623 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:03:01,841 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} notificationworker stdout | 2023-08-08 06:03:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:03:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:05.371682+00:00 (in 3.192619 seconds) notificationworker stdout | 2023-08-08 06:03:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:02 UTC)" (scheduled at 2023-08-08 06:03:02.178530+00:00) notificationworker stdout | 2023-08-08 06:03:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:03:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:02 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:03:02,229 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 06:03:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:03:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:12.293757+00:00 (in 9.999499 seconds) chunkcleanupworker stdout | 2023-08-08 06:03:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:12 UTC)" (scheduled at 2023-08-08 06:03:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:03:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:03:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 2, 294823), True, datetime.datetime(2023, 8, 8, 6, 3, 2, 294823), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:03:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:03:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:03:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:03:03,142 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:03:03,145 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:03:03,149 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:03:03,152 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:03:03,156 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:03:03,159 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:03:03,167 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:03:03,171 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:03:03,174 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:03:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:03:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:09.793658+00:00 (in 5.605693 seconds) repositorygcworker stdout | 2023-08-08 06:03:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:04 UTC)" (scheduled at 2023-08-08 06:03:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:03:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:03:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:03:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:03:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:15.371682+00:00 (in 9.999494 seconds) notificationworker stdout | 2023-08-08 06:03:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:15 UTC)" (scheduled at 2023-08-08 06:03:05.371682+00:00) notificationworker stdout | 2023-08-08 06:03:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:03:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 5, 373005), True, datetime.datetime(2023, 8, 8, 6, 3, 5, 373005), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:03:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:03:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:03:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:03:07,179 [198] [DEBUG] [app] Starting request: urn:request:daf49dd5-fc2f-40ef-8f67-4060a662c7ec (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:03:07,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:07,180 [197] [DEBUG] [app] Starting request: urn:request:68f559b8-30c9-4c57-b821-d9ca5d9064f9 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:03:07,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:07,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:03:07,185 [207] [DEBUG] [app] Starting request: urn:request:a581b499-1f43-4b15-a1d7-9fd11d18a07a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:03:07,185 [207] [DEBUG] [app] Ending request: urn:request:a581b499-1f43-4b15-a1d7-9fd11d18a07a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a581b499-1f43-4b15-a1d7-9fd11d18a07a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:03:07,186 [206] [DEBUG] [app] Starting request: urn:request:409adc92-2306-4acb-aa7e-701c8a1e9a5e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:03:07,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:03:07,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:03:07,186 [206] [DEBUG] [app] Ending request: urn:request:409adc92-2306-4acb-aa7e-701c8a1e9a5e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:409adc92-2306-4acb-aa7e-701c8a1e9a5e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:03:07,187 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:03:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:07,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:07,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:07,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:07,191 [197] [DEBUG] [app] Starting request: urn:request:b6eac40d-10ec-49c8-96b5-1036bcbf7213 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:03:07,191 [197] [DEBUG] [app] Ending request: urn:request:b6eac40d-10ec-49c8-96b5-1036bcbf7213 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b6eac40d-10ec-49c8-96b5-1036bcbf7213', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:03:07,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:03:07,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:03:07,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:07,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:03:07,193 [199] [DEBUG] [app] Starting request: urn:request:94e2b3fe-bb57-4ee7-a694-bfee012c0ed8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:03:07,194 [199] [DEBUG] [app] Ending request: urn:request:94e2b3fe-bb57-4ee7-a694-bfee012c0ed8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:94e2b3fe-bb57-4ee7-a694-bfee012c0ed8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:03:07,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:03:07,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:07,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:03:07,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:03:07,199 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:03:07,199 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:37.197412+00:00 (in 29.997622 seconds) buildlogsarchiver stdout | 2023-08-08 06:03:07,200 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:37 UTC)" (scheduled at 2023-08-08 06:03:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:03:07,200 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 3, 7, 200253), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:03:07,202 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:03:07,204 [198] [DEBUG] [app] Ending request: urn:request:daf49dd5-fc2f-40ef-8f67-4060a662c7ec (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:daf49dd5-fc2f-40ef-8f67-4060a662c7ec', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:03:07,205 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:03:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:03:07,205 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:03:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:03:07,207 [197] [DEBUG] [app] Ending request: urn:request:68f559b8-30c9-4c57-b821-d9ca5d9064f9 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:68f559b8-30c9-4c57-b821-d9ca5d9064f9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:03:07,207 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:03:07,207 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:03:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:03:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) buildlogsarchiver stdout | 2023-08-08 06:03:07,208 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:03:07,209 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:03:07,210 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:03:07,216 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:03:07,216 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:03:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:03:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:19.417660+00:00 (in 9.999445 seconds) gcworker stdout | 2023-08-08 06:03:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:19 UTC)" (scheduled at 2023-08-08 06:03:09.417660+00:00) gcworker stdout | 2023-08-08 06:03:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264989418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:03:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:03:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:03:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:03:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:03:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:04.187429+00:00 (in 54.393217 seconds) repositorygcworker stdout | 2023-08-08 06:03:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:09 UTC)" (scheduled at 2023-08-08 06:03:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:03:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:03:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 9, 795271), True, datetime.datetime(2023, 8, 8, 6, 3, 9, 795271), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:03:09,804 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:03:09,804 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:03:09,804 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:09 UTC)" executed successfully builder stdout | 2023-08-08 06:03:11,633 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:03:11,642 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:03:11,642 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:03:11,647 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:03:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:03:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:22.293757+00:00 (in 9.999489 seconds) chunkcleanupworker stdout | 2023-08-08 06:03:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:22 UTC)" (scheduled at 2023-08-08 06:03:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:03:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:03:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 12, 294769), True, datetime.datetime(2023, 8, 8, 6, 3, 12, 294769), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:03:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:03:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:03:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:03:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:03:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:54.284093+00:00 (in 41.186161 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:03:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:13 UTC)" (scheduled at 2023-08-08 06:03:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:03:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:03:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 13, 98441), True, datetime.datetime(2023, 8, 8, 6, 3, 13, 98441), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:03:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:03:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:03:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:03:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:03:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:25.371682+00:00 (in 9.999031 seconds) notificationworker stdout | 2023-08-08 06:03:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:25 UTC)" (scheduled at 2023-08-08 06:03:15.371682+00:00) notificationworker stdout | 2023-08-08 06:03:15,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:03:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 15, 373057), True, datetime.datetime(2023, 8, 8, 6, 3, 15, 373057), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:03:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:03:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:03:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:03:17,625 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:03:18,841 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:03:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:03:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:29.417660+00:00 (in 9.999518 seconds) gcworker stdout | 2023-08-08 06:03:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:29 UTC)" (scheduled at 2023-08-08 06:03:19.417660+00:00) gcworker stdout | 2023-08-08 06:03:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690264999418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:03:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:03:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:03:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:03:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:03:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:26.890741+00:00 (in 7.202890 seconds) securityworker stdout | 2023-08-08 06:03:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:49 UTC)" (scheduled at 2023-08-08 06:03:19.687363+00:00) securityworker stdout | 2023-08-08 06:03:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:03:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:03:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:03:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:03:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:03:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:03:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 58, 19, 692163), 1, 2]) securityworker stdout | 2023-08-08 06:03:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 58, 19, 692163), 1, 2]) securityworker stdout | 2023-08-08 06:03:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:03:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:03:19,985 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:03:20,434 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:03:21,657 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:03:21,667 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:03:21,667 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:03:21,672 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:03:22,178 [197] [DEBUG] [app] Starting request: urn:request:1d72d501-9498-46a3-ba46-c874307f100f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:03:22,179 [198] [DEBUG] [app] Starting request: urn:request:47c53fb3-8510-425a-aacf-c7e08494492a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:03:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:22,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:22,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:03:22,184 [207] [DEBUG] [app] Starting request: urn:request:2c90335e-4d5d-4943-9817-45eff1fc26f1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:03:22,184 [207] [DEBUG] [app] Ending request: urn:request:2c90335e-4d5d-4943-9817-45eff1fc26f1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2c90335e-4d5d-4943-9817-45eff1fc26f1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:03:22,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:03:22,185 [203] [DEBUG] [app] Starting request: urn:request:cc7836a1-a285-42d3-b4d9-23320775a2cd (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:03:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:03:22,185 [203] [DEBUG] [app] Ending request: urn:request:cc7836a1-a285-42d3-b4d9-23320775a2cd (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:cc7836a1-a285-42d3-b4d9-23320775a2cd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:03:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:03:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:22,190 [198] [DEBUG] [app] Starting request: urn:request:4c4f83e1-3cc0-46dc-81fd-9977711de03d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:03:22,190 [198] [DEBUG] [app] Ending request: urn:request:4c4f83e1-3cc0-46dc-81fd-9977711de03d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4c4f83e1-3cc0-46dc-81fd-9977711de03d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:03:22,190 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:03:22,190 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:22,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:03:22,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:22,192 [200] [DEBUG] [app] Starting request: urn:request:76b2ee4e-6457-4ba7-9bfc-0ae7b841873e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:03:22,192 [200] [DEBUG] [app] Ending request: urn:request:76b2ee4e-6457-4ba7-9bfc-0ae7b841873e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:76b2ee4e-6457-4ba7-9bfc-0ae7b841873e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:03:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:03:22,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:22,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:03:22,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:03:22,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:03:22,202 [197] [DEBUG] [app] Ending request: urn:request:1d72d501-9498-46a3-ba46-c874307f100f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1d72d501-9498-46a3-ba46-c874307f100f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:03:22,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:03:22,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:03:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:03:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:03:22,203 [198] [DEBUG] [app] Ending request: urn:request:47c53fb3-8510-425a-aacf-c7e08494492a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:47c53fb3-8510-425a-aacf-c7e08494492a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:03:22,203 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:03:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:03:22,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:03:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:03:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:03:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:28.874906+00:00 (in 6.580590 seconds) chunkcleanupworker stdout | 2023-08-08 06:03:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:32 UTC)" (scheduled at 2023-08-08 06:03:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:03:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:03:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 22, 294893), True, datetime.datetime(2023, 8, 8, 6, 3, 22, 294893), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:03:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:03:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:03:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:03:23,156 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:03:24,902 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:03:24,906 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:03:24,909 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:03:25,042 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:03:25,158 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:03:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:03:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:35.371682+00:00 (in 9.999437 seconds) notificationworker stdout | 2023-08-08 06:03:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:35 UTC)" (scheduled at 2023-08-08 06:03:25.371682+00:00) notificationworker stdout | 2023-08-08 06:03:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:03:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 25, 372605), True, datetime.datetime(2023, 8, 8, 6, 3, 25, 372605), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:03:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:03:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:03:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:35 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:03:25,387 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} exportactionlogsworker stdout | 2023-08-08 06:03:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:03:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:33.514094+00:00 (in 7.999196 seconds) exportactionlogsworker stdout | 2023-08-08 06:03:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:25 UTC)" (scheduled at 2023-08-08 06:03:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:03:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:03:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:03:26,494 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:03:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:03:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:49.687363+00:00 (in 22.796070 seconds) securityworker stdout | 2023-08-08 06:03:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:56 UTC)" (scheduled at 2023-08-08 06:03:26.890741+00:00) securityworker stdout | 2023-08-08 06:03:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:03:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:03:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:03:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:03:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:03:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:03:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:03:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:03:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:03:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:03:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 58, 26, 896567), 1, 2]) securityworker stdout | 2023-08-08 06:03:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 58, 26, 896567), 1, 2]) securityworker stdout | 2023-08-08 06:03:26,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:26,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:26,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:03:26,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:03:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:03:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:03:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:03:26,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:03:26,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:03:26,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:03:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:03:27,030 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:03:27,146 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:03:27,308 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:03:27,360 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:03:27,377 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:03:27,395 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:03:27,398 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:03:28,434 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:03:28,610 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 06:03:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:03:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:32.293757+00:00 (in 3.418159 seconds) chunkcleanupworker stdout | 2023-08-08 06:03:28,876 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:28 UTC)" (scheduled at 2023-08-08 06:03:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:03:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:03:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:03:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:03:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:39.417660+00:00 (in 9.999442 seconds) gcworker stdout | 2023-08-08 06:03:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:39 UTC)" (scheduled at 2023-08-08 06:03:29.417660+00:00) gcworker stdout | 2023-08-08 06:03:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265009418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:03:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:03:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:03:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:03:29,663 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:03:30,514 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:03:30,614 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:03:31,022 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:03:31,595 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} builder stdout | 2023-08-08 06:03:31,681 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:03:31,690 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:03:31,690 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:03:31,694 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:03:31,877 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:03:32,258 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 06:03:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:03:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:42.293757+00:00 (in 9.999423 seconds) chunkcleanupworker stdout | 2023-08-08 06:03:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:42 UTC)" (scheduled at 2023-08-08 06:03:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:03:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:03:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 32, 294874), True, datetime.datetime(2023, 8, 8, 6, 3, 32, 294874), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:03:32,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:03:32,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:03:32,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:42 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:03:33,151 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:03:33,157 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:03:33,161 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:03:33,164 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:03:33,167 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:03:33,170 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:03:33,176 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:03:33,181 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:03:33,184 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} exportactionlogsworker stdout | 2023-08-08 06:03:33,515 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:03:33,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:47.398551+00:00 (in 13.883045 seconds) exportactionlogsworker stdout | 2023-08-08 06:03:33,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:08:33 UTC)" (scheduled at 2023-08-08 06:03:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:03:33,516 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 3, 33, 515976), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:03:33,523 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 3, 33, 515976), True, datetime.datetime(2023, 8, 8, 6, 3, 33, 515976), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:03:33,525 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 3, 33, 515976), True, datetime.datetime(2023, 8, 8, 6, 3, 33, 515976), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 3, 33, 515976), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:03:33,526 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:03:33,526 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:08:33 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:03:34,946 [200] [DEBUG] [app] Starting request: urn:request:eb73e551-f3d1-4115-8eba-62eb90adb683 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:03:34,947 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:34,947 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:34,947 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:03:34,955 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:03:34,955 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:03:34,955 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:34,955 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 06:03:34,955 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:34,955 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:03:34,955 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:03:34,956 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:03:34,958 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:03:34,965 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:03:34,965 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:03:34,969 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 06:03:34,973 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:34,974 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:34,974 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:34,975 [200] [DEBUG] [app] Ending request: urn:request:eb73e551-f3d1-4115-8eba-62eb90adb683 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status) {'endpoint': 'api.repositorybuildstatus', 'request_id': 'urn:request:eb73e551-f3d1-4115-8eba-62eb90adb683', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:03:34,975 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:03:34,976 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:03:34 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/1.0" 200 2082 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:03:34 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/status HTTP/2.0" 200 2082 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 912 0.031) gunicorn-web stdout | 2023-08-08 06:03:35,282 [198] [DEBUG] [app] Starting request: urn:request:a255a637-2cfb-4c67-89e8-513f289703c2 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:03:35,282 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:35,282 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:35,283 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:03:35,290 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:03:35,290 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:03:35,290 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:35,290 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2023-08-08 06:03:35,290 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:35,290 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:03:35,290 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:03:35,290 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:03:35,293 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:03:35,299 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:03:35,300 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:03:35,302 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:03:35,304 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE ("t1"."uuid" = %s) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['d5b40e12-f92c-42e5-9c25-9e76946bc5a3', 1, 0]) gunicorn-web stdout | 2023-08-08 06:03:35,310 [198] [DEBUG] [app] Ending request: urn:request:a255a637-2cfb-4c67-89e8-513f289703c2 (/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs) {'endpoint': 'api.repositorybuildlogs', 'request_id': 'urn:request:a255a637-2cfb-4c67-89e8-513f289703c2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1', 'path': '/api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs', 'parameters': {'start': '1'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:03:35,310 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:03:35,311 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:03:35 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/1.0" 200 11210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:03:35 +0000] "GET /api/v1/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3/logs?start=1 HTTP/2.0" 200 11210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 447 0.030) notificationworker stdout | 2023-08-08 06:03:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:03:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:45.371682+00:00 (in 9.999482 seconds) notificationworker stdout | 2023-08-08 06:03:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:45 UTC)" (scheduled at 2023-08-08 06:03:35.371682+00:00) notificationworker stdout | 2023-08-08 06:03:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:03:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 35, 372587), True, datetime.datetime(2023, 8, 8, 6, 3, 35, 372587), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:03:35,379 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:03:35,379 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:03:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:03:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:03:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:54.393627+00:00 (in 17.813195 seconds) namespacegcworker stdout | 2023-08-08 06:03:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:36 UTC)" (scheduled at 2023-08-08 06:03:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:03:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:03:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:03:37,178 [199] [DEBUG] [app] Starting request: urn:request:cf80c94e-a5fd-4678-849a-1515d54ed40d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:03:37,179 [197] [DEBUG] [app] Starting request: urn:request:a94d5c43-6663-4264-98df-c47561309ef5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:03:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:03:37,184 [203] [DEBUG] [app] Starting request: urn:request:b26ccf44-bd30-401c-ab0e-98ec83bbd25c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:03:37,185 [209] [DEBUG] [app] Starting request: urn:request:905eca93-75b4-4cd5-9b77-60bf84c127c4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:03:37,185 [203] [DEBUG] [app] Ending request: urn:request:b26ccf44-bd30-401c-ab0e-98ec83bbd25c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b26ccf44-bd30-401c-ab0e-98ec83bbd25c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:03:37,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:03:37,185 [209] [DEBUG] [app] Ending request: urn:request:905eca93-75b4-4cd5-9b77-60bf84c127c4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:905eca93-75b4-4cd5-9b77-60bf84c127c4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:03:37,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:03:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:03:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:37,190 [199] [DEBUG] [app] Starting request: urn:request:2b9591e6-8176-490a-a8fa-c9a39f24142c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:03:37,191 [198] [DEBUG] [app] Starting request: urn:request:c8a9d704-b693-4fb9-a0ed-aef73ac6ea71 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:03:37,191 [199] [DEBUG] [app] Ending request: urn:request:2b9591e6-8176-490a-a8fa-c9a39f24142c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2b9591e6-8176-490a-a8fa-c9a39f24142c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:03:37,191 [198] [DEBUG] [app] Ending request: urn:request:c8a9d704-b693-4fb9-a0ed-aef73ac6ea71 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c8a9d704-b693-4fb9-a0ed-aef73ac6ea71', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:03:37,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:03:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:03:37,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:37,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:03:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:03:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:03:37,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:03:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:03:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:07.197412+00:00 (in 29.999544 seconds) buildlogsarchiver stdout | 2023-08-08 06:03:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:07 UTC)" (scheduled at 2023-08-08 06:03:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:03:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 3, 37, 198268), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:03:37,202 [199] [DEBUG] [app] Ending request: urn:request:cf80c94e-a5fd-4678-849a-1515d54ed40d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cf80c94e-a5fd-4678-849a-1515d54ed40d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:03:37,202 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:03:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:03:37,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:03:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:03:37,203 [197] [DEBUG] [app] Ending request: urn:request:a94d5c43-6663-4264-98df-c47561309ef5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a94d5c43-6663-4264-98df-c47561309ef5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:03:37,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:03:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:03:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:03:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:03:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:03:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:03:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:03:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:03:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:03:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:03:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:49.417660+00:00 (in 9.999509 seconds) gcworker stdout | 2023-08-08 06:03:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:49 UTC)" (scheduled at 2023-08-08 06:03:39.417660+00:00) gcworker stdout | 2023-08-08 06:03:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265019418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:03:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:03:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:03:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:49 UTC)" executed successfully builder stdout | 2023-08-08 06:03:41,704 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:03:41,714 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:03:41,714 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:03:41,719 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:03:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:03:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:52.293757+00:00 (in 9.999480 seconds) chunkcleanupworker stdout | 2023-08-08 06:03:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:52 UTC)" (scheduled at 2023-08-08 06:03:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:03:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:03:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 42, 294793), True, datetime.datetime(2023, 8, 8, 6, 3, 42, 294793), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:03:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:03:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:03:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:03:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:03:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:49.178248+00:00 (in 3.806014 seconds) notificationworker stdout | 2023-08-08 06:03:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:55 UTC)" (scheduled at 2023-08-08 06:03:45.371682+00:00) notificationworker stdout | 2023-08-08 06:03:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:03:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 45, 372751), True, datetime.datetime(2023, 8, 8, 6, 3, 45, 372751), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:03:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:03:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:03:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:03:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:03:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:25.514341+00:00 (in 38.115253 seconds) exportactionlogsworker stdout | 2023-08-08 06:03:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:47 UTC)" (scheduled at 2023-08-08 06:03:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:03:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:03:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 47, 399559), True, datetime.datetime(2023, 8, 8, 6, 3, 47, 399559), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:03:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:03:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:03:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:03:47,663 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:03:48,878 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} notificationworker stdout | 2023-08-08 06:03:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:03:49,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:55.371682+00:00 (in 6.192837 seconds) notificationworker stdout | 2023-08-08 06:03:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:08:49 UTC)" (scheduled at 2023-08-08 06:03:49.178248+00:00) notificationworker stdout | 2023-08-08 06:03:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 3, 49, 179316), 'notification/%']) notificationworker stdout | 2023-08-08 06:03:49,186 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 3, 49, 179316), True, datetime.datetime(2023, 8, 8, 6, 3, 49, 179316), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:03:49,188 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 3, 49, 179316), True, datetime.datetime(2023, 8, 8, 6, 3, 49, 179316), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 3, 49, 179316), 'notification/%']) notificationworker stdout | 2023-08-08 06:03:49,190 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:03:49,190 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:08:49 UTC)" executed successfully gcworker stdout | 2023-08-08 06:03:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:03:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:59.417660+00:00 (in 9.999508 seconds) gcworker stdout | 2023-08-08 06:03:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:59 UTC)" (scheduled at 2023-08-08 06:03:49.417660+00:00) gcworker stdout | 2023-08-08 06:03:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265029418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:03:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:03:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:03:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:03:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:03:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:03:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:03:56.890741+00:00 (in 7.202903 seconds) securityworker stdout | 2023-08-08 06:03:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:19 UTC)" (scheduled at 2023-08-08 06:03:49.687363+00:00) securityworker stdout | 2023-08-08 06:03:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:03:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:03:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:03:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:03:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:03:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:03:50,023 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:03:50,456 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:03:51,720 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:03:51,730 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:03:51,730 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:03:51,734 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:03:52,178 [199] [DEBUG] [app] Starting request: urn:request:8aaef3ff-5b21-4b1e-bb83-8569baffb19e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:03:52,179 [197] [DEBUG] [app] Starting request: urn:request:448a4ea0-cd0e-4ebb-89cf-3518397fafb0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:03:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:03:52,185 [203] [DEBUG] [app] Starting request: urn:request:925b16c5-edfa-48e9-8da4-56d481528479 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:03:52,186 [203] [DEBUG] [app] Ending request: urn:request:925b16c5-edfa-48e9-8da4-56d481528479 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:925b16c5-edfa-48e9-8da4-56d481528479', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:03:52,186 [210] [DEBUG] [app] Starting request: urn:request:fbf0e0c6-d407-4b85-9adc-70cf0e69aeec (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:03:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:03:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:03:52,186 [210] [DEBUG] [app] Ending request: urn:request:fbf0e0c6-d407-4b85-9adc-70cf0e69aeec (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fbf0e0c6-d407-4b85-9adc-70cf0e69aeec', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:03:52,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:03:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:52,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:52,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:03:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:52,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:03:52,191 [200] [DEBUG] [app] Starting request: urn:request:b82ee84a-e795-4f76-bd63-ca1fb51859e4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:03:52,192 [200] [DEBUG] [app] Ending request: urn:request:b82ee84a-e795-4f76-bd63-ca1fb51859e4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b82ee84a-e795-4f76-bd63-ca1fb51859e4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:03:52,192 [199] [DEBUG] [app] Starting request: urn:request:aa064b1c-fcc0-4434-b8e9-dd512de407b6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:03:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:03:52,192 [199] [DEBUG] [app] Ending request: urn:request:aa064b1c-fcc0-4434-b8e9-dd512de407b6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:aa064b1c-fcc0-4434-b8e9-dd512de407b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:03:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:03:52,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:03:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:03:52,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:52,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:03:52,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:03:52,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:03:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:03:52,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:03:52,204 [197] [DEBUG] [app] Ending request: urn:request:448a4ea0-cd0e-4ebb-89cf-3518397fafb0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:448a4ea0-cd0e-4ebb-89cf-3518397fafb0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:03:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:03:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:03:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:03:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:03:52,205 [199] [DEBUG] [app] Ending request: urn:request:8aaef3ff-5b21-4b1e-bb83-8569baffb19e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8aaef3ff-5b21-4b1e-bb83-8569baffb19e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:03:52,205 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:03:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) gunicorn-web stdout | 2023-08-08 06:03:52,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:03:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:03:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:03:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:02.293757+00:00 (in 9.999490 seconds) chunkcleanupworker stdout | 2023-08-08 06:03:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:02 UTC)" (scheduled at 2023-08-08 06:03:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:03:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:03:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 52, 294656), True, datetime.datetime(2023, 8, 8, 6, 3, 52, 294656), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:03:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:03:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:03:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:03:53,193 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:03:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:03:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:13.097375+00:00 (in 18.812723 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:03:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:54 UTC)" (scheduled at 2023-08-08 06:03:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:03:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:03:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:03:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:03:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:36.579841+00:00 (in 42.185646 seconds) namespacegcworker stdout | 2023-08-08 06:03:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:54 UTC)" (scheduled at 2023-08-08 06:03:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:03:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:03:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 54, 394700), True, datetime.datetime(2023, 8, 8, 6, 3, 54, 394700), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:03:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:03:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:03:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:04:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:03:54,910 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:03:54,914 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:03:54,917 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:03:55,065 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:03:55,179 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:03:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:03:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:02.178530+00:00 (in 6.806345 seconds) notificationworker stdout | 2023-08-08 06:03:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:05 UTC)" (scheduled at 2023-08-08 06:03:55.371682+00:00) notificationworker stdout | 2023-08-08 06:03:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:03:55,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 3, 55, 372768), True, datetime.datetime(2023, 8, 8, 6, 3, 55, 372768), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:03:55,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:03:55,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:03:55,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:05 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:03:55,423 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:03:56,531 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:03:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:03:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:19.687363+00:00 (in 22.796099 seconds) securityworker stdout | 2023-08-08 06:03:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:26 UTC)" (scheduled at 2023-08-08 06:03:56.890741+00:00) securityworker stdout | 2023-08-08 06:03:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:03:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:03:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:03:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:03:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:03:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:03:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:03:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:03:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:03:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:03:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 58, 56, 896472), 1, 2]) securityworker stdout | 2023-08-08 06:03:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 58, 56, 896472), 1, 2]) securityworker stdout | 2023-08-08 06:03:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:03:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:03:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:03:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:03:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:03:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:03:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:03:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:03:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:03:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:03:57,067 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:03:57,156 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:03:57,316 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:03:57,371 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:03:57,388 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:03:57,407 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:03:57,415 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:03:58,471 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:03:58,633 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:03:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:03:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:09.417660+00:00 (in 9.999503 seconds) gcworker stdout | 2023-08-08 06:03:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:09 UTC)" (scheduled at 2023-08-08 06:03:59.417660+00:00) gcworker stdout | 2023-08-08 06:03:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265039418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:03:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:03:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:03:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:03:59,700 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:04:00,552 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:04:00,625 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:04:01,047 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:04:01,618 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} builder stdout | 2023-08-08 06:04:01,745 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:04:01,755 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:04:01,755 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:04:01,759 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:04:01,910 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} notificationworker stdout | 2023-08-08 06:04:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:04:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:05.371682+00:00 (in 3.192640 seconds) notificationworker stdout | 2023-08-08 06:04:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:02 UTC)" (scheduled at 2023-08-08 06:04:02.178530+00:00) notificationworker stdout | 2023-08-08 06:04:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:04:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:02 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:04:02,272 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} chunkcleanupworker stdout | 2023-08-08 06:04:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:04:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:12.293757+00:00 (in 9.999522 seconds) chunkcleanupworker stdout | 2023-08-08 06:04:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:12 UTC)" (scheduled at 2023-08-08 06:04:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:04:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:04:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 2, 294590), True, datetime.datetime(2023, 8, 8, 6, 4, 2, 294590), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:04:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:04:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:12 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:04:03,161 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:04:03,166 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:04:03,171 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:04:03,174 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:04:03,177 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:04:03,181 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:04:03,185 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:04:03,189 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:04:03,193 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:04:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:04:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:09.793658+00:00 (in 5.605612 seconds) repositorygcworker stdout | 2023-08-08 06:04:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:04 UTC)" (scheduled at 2023-08-08 06:04:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:04:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:04:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:04:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:04:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:15.371682+00:00 (in 9.999509 seconds) notificationworker stdout | 2023-08-08 06:04:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:15 UTC)" (scheduled at 2023-08-08 06:04:05.371682+00:00) notificationworker stdout | 2023-08-08 06:04:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:04:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 5, 372787), True, datetime.datetime(2023, 8, 8, 6, 4, 5, 372787), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:04:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:04:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:04:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:04:07,178 [198] [DEBUG] [app] Starting request: urn:request:984d5ba2-9b00-4d8c-a2f2-20142b760f14 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:04:07,178 [197] [DEBUG] [app] Starting request: urn:request:3263c816-f4a3-46e8-9d21-4ed326cfa6f9 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:04:07,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:07,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:04:07,184 [208] [DEBUG] [app] Starting request: urn:request:7f76e924-b147-4eb0-9cd2-b93e1327b5a3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:04:07,184 [204] [DEBUG] [app] Starting request: urn:request:d8b8cfd1-70c0-4252-ae56-2f480b30f032 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:04:07,185 [208] [DEBUG] [app] Ending request: urn:request:7f76e924-b147-4eb0-9cd2-b93e1327b5a3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7f76e924-b147-4eb0-9cd2-b93e1327b5a3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:04:07,185 [204] [DEBUG] [app] Ending request: urn:request:d8b8cfd1-70c0-4252-ae56-2f480b30f032 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d8b8cfd1-70c0-4252-ae56-2f480b30f032', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:04:07,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:04:07,185 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:07,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:07,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:07,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:07,191 [197] [DEBUG] [app] Starting request: urn:request:a69792bf-1949-4afe-913d-16ff4eee23a5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:04:07,192 [197] [DEBUG] [app] Ending request: urn:request:a69792bf-1949-4afe-913d-16ff4eee23a5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a69792bf-1949-4afe-913d-16ff4eee23a5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:04:07,192 [200] [DEBUG] [app] Starting request: urn:request:d717a4b1-1f23-4914-a30e-4f212f269548 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:04:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:07,192 [200] [DEBUG] [app] Ending request: urn:request:d717a4b1-1f23-4914-a30e-4f212f269548 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d717a4b1-1f23-4914-a30e-4f212f269548', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:04:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:04:07,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:07,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:04:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:04:07,198 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:04:07,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:04:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:37.197412+00:00 (in 29.998993 seconds) buildlogsarchiver stdout | 2023-08-08 06:04:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:37 UTC)" (scheduled at 2023-08-08 06:04:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:04:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 4, 7, 198827), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:04:07,203 [197] [DEBUG] [app] Ending request: urn:request:3263c816-f4a3-46e8-9d21-4ed326cfa6f9 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3263c816-f4a3-46e8-9d21-4ed326cfa6f9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:04:07,204 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:04:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:04:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:04:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:04:07,204 [198] [DEBUG] [app] Ending request: urn:request:984d5ba2-9b00-4d8c-a2f2-20142b760f14 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:984d5ba2-9b00-4d8c-a2f2-20142b760f14', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:04:07,204 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:04:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) gunicorn-web stdout | 2023-08-08 06:04:07,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:04:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:04:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:04:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:04:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:04:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:04:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:04:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:04:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:19.417660+00:00 (in 9.999506 seconds) gcworker stdout | 2023-08-08 06:04:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:19 UTC)" (scheduled at 2023-08-08 06:04:09.417660+00:00) gcworker stdout | 2023-08-08 06:04:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265049418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:04:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:04:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:04:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:04:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:04:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:19.187230+00:00 (in 9.393034 seconds) repositorygcworker stdout | 2023-08-08 06:04:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:09 UTC)" (scheduled at 2023-08-08 06:04:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:04:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:04:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 9, 795173), True, datetime.datetime(2023, 8, 8, 6, 4, 9, 795173), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:04:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:04:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:04:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:09 UTC)" executed successfully builder stdout | 2023-08-08 06:04:11,763 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:04:11,773 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:04:11,773 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:04:11,778 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:04:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:22.293757+00:00 (in 9.999487 seconds) chunkcleanupworker stdout | 2023-08-08 06:04:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:22 UTC)" (scheduled at 2023-08-08 06:04:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:04:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:04:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 12, 294679), True, datetime.datetime(2023, 8, 8, 6, 4, 12, 294679), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:04:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:04:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:12,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:04:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:04:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:54.284093+00:00 (in 41.186170 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:04:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:13 UTC)" (scheduled at 2023-08-08 06:04:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:04:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:04:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 13, 98498), True, datetime.datetime(2023, 8, 8, 6, 4, 13, 98498), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:04:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:04:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:04:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:04:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:04:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:25.371682+00:00 (in 9.999470 seconds) notificationworker stdout | 2023-08-08 06:04:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:25 UTC)" (scheduled at 2023-08-08 06:04:15.371682+00:00) notificationworker stdout | 2023-08-08 06:04:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:04:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 15, 372697), True, datetime.datetime(2023, 8, 8, 6, 4, 15, 372697), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:04:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:04:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:04:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:04:17,693 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:04:18,916 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} repositorygcworker stdout | 2023-08-08 06:04:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:04:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:04.187429+00:00 (in 44.999688 seconds) repositorygcworker stdout | 2023-08-08 06:04:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:09:19 UTC)" (scheduled at 2023-08-08 06:04:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:04:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 4, 19, 188180), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:04:19,196 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 4, 19, 188180), True, datetime.datetime(2023, 8, 8, 6, 4, 19, 188180), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:04:19,198 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 4, 19, 188180), True, datetime.datetime(2023, 8, 8, 6, 4, 19, 188180), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 4, 19, 188180), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:04:19,199 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:04:19,199 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:09:19 UTC)" executed successfully gcworker stdout | 2023-08-08 06:04:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:04:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:29.417660+00:00 (in 9.999503 seconds) gcworker stdout | 2023-08-08 06:04:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:29 UTC)" (scheduled at 2023-08-08 06:04:19.417660+00:00) gcworker stdout | 2023-08-08 06:04:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265059418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:04:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:04:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:04:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:04:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:04:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:26.890741+00:00 (in 7.202841 seconds) securityworker stdout | 2023-08-08 06:04:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:49 UTC)" (scheduled at 2023-08-08 06:04:19.687363+00:00) securityworker stdout | 2023-08-08 06:04:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:04:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:04:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:04:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:04:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:04:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:04:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 59, 19, 692360), 1, 2]) securityworker stdout | 2023-08-08 06:04:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 59, 19, 692360), 1, 2]) securityworker stdout | 2023-08-08 06:04:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:04:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:04:20,031 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:04:20,486 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:04:21,788 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:04:21,797 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:04:21,797 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:04:21,801 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:04:22,178 [197] [DEBUG] [app] Starting request: urn:request:d012fa40-121d-4565-bfc6-b9a135f659ac (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:04:22,179 [200] [DEBUG] [app] Starting request: urn:request:bd629725-7a95-4b3d-9932-75e27464779a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:04:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:22,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:04:22,184 [208] [DEBUG] [app] Starting request: urn:request:ea10f02e-bf0a-4b85-ba92-3de2a8dc432e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:04:22,184 [209] [DEBUG] [app] Starting request: urn:request:d6f952f2-cc0a-413d-a97c-492af948e364 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:04:22,184 [208] [DEBUG] [app] Ending request: urn:request:ea10f02e-bf0a-4b85-ba92-3de2a8dc432e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ea10f02e-bf0a-4b85-ba92-3de2a8dc432e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:04:22,185 [209] [DEBUG] [app] Ending request: urn:request:d6f952f2-cc0a-413d-a97c-492af948e364 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d6f952f2-cc0a-413d-a97c-492af948e364', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:04:22,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:04:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:04:22,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:22,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:22,190 [198] [DEBUG] [app] Starting request: urn:request:b327d75f-5788-473d-a2dd-c3895d066197 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:04:22,191 [197] [DEBUG] [app] Starting request: urn:request:31930293-a757-4988-959a-9a602e93be1b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:04:22,191 [198] [DEBUG] [app] Ending request: urn:request:b327d75f-5788-473d-a2dd-c3895d066197 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b327d75f-5788-473d-a2dd-c3895d066197', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:04:22,191 [197] [DEBUG] [app] Ending request: urn:request:31930293-a757-4988-959a-9a602e93be1b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:31930293-a757-4988-959a-9a602e93be1b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:04:22,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:04:22,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:04:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:04:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:04:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:04:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:04:22,203 [197] [DEBUG] [app] Ending request: urn:request:d012fa40-121d-4565-bfc6-b9a135f659ac (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d012fa40-121d-4565-bfc6-b9a135f659ac', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:04:22,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:04:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:04:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:04:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:04:22,204 [200] [DEBUG] [app] Ending request: urn:request:bd629725-7a95-4b3d-9932-75e27464779a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bd629725-7a95-4b3d-9932-75e27464779a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:04:22,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:04:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:04:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:04:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:04:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:04:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:28.874906+00:00 (in 6.580625 seconds) chunkcleanupworker stdout | 2023-08-08 06:04:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:32 UTC)" (scheduled at 2023-08-08 06:04:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:04:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:04:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 22, 294666), True, datetime.datetime(2023, 8, 8, 6, 4, 22, 294666), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:04:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:04:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:04:23,209 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:04:24,919 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:04:24,924 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:04:24,929 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:04:25,073 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:04:25,217 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:04:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:04:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:35.371682+00:00 (in 9.999483 seconds) notificationworker stdout | 2023-08-08 06:04:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:35 UTC)" (scheduled at 2023-08-08 06:04:25.371682+00:00) notificationworker stdout | 2023-08-08 06:04:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:04:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 25, 372774), True, datetime.datetime(2023, 8, 8, 6, 4, 25, 372774), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:04:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:04:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:04:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:35 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:04:25,442 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} exportactionlogsworker stdout | 2023-08-08 06:04:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:04:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:47.398551+00:00 (in 21.883673 seconds) exportactionlogsworker stdout | 2023-08-08 06:04:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:25 UTC)" (scheduled at 2023-08-08 06:04:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:04:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:04:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:04:26,545 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:04:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:04:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:49.687363+00:00 (in 22.796092 seconds) securityworker stdout | 2023-08-08 06:04:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:56 UTC)" (scheduled at 2023-08-08 06:04:26.890741+00:00) securityworker stdout | 2023-08-08 06:04:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:04:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:04:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:04:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:04:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:04:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:04:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:04:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:04:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:04:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:04:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 59, 26, 896482), 1, 2]) securityworker stdout | 2023-08-08 06:04:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 59, 26, 896482), 1, 2]) securityworker stdout | 2023-08-08 06:04:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:04:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:04:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:04:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:04:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:04:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:04:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:04:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:04:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:04:27,086 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:04:27,169 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:04:27,326 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:04:27,381 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:04:27,403 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:04:27,421 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:04:27,426 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:04:28,502 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:04:28,656 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 06:04:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:04:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:32.293757+00:00 (in 3.418301 seconds) chunkcleanupworker stdout | 2023-08-08 06:04:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:28 UTC)" (scheduled at 2023-08-08 06:04:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:04:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:04:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:04:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:04:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:39.417660+00:00 (in 9.999452 seconds) gcworker stdout | 2023-08-08 06:04:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:39 UTC)" (scheduled at 2023-08-08 06:04:29.417660+00:00) gcworker stdout | 2023-08-08 06:04:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265069418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:04:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:04:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:04:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:04:29,737 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:04:30,591 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:04:30,662 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:04:31,074 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:04:31,629 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} builder stdout | 2023-08-08 06:04:31,805 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:04:31,816 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:04:31,816 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:04:31,820 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:04:31,948 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} chunkcleanupworker stdout | 2023-08-08 06:04:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:04:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:42.293757+00:00 (in 9.999400 seconds) chunkcleanupworker stdout | 2023-08-08 06:04:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:42 UTC)" (scheduled at 2023-08-08 06:04:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:04:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:04:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 32, 294949), True, datetime.datetime(2023, 8, 8, 6, 4, 32, 294949), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:04:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:04:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:42 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:04:32,308 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:04:33,170 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:04:33,174 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:04:33,181 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:04:33,184 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:04:33,189 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:04:33,193 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:04:33,195 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:04:33,200 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:04:33,202 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:04:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:04:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:45.371682+00:00 (in 9.999466 seconds) notificationworker stdout | 2023-08-08 06:04:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:45 UTC)" (scheduled at 2023-08-08 06:04:35.371682+00:00) notificationworker stdout | 2023-08-08 06:04:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:04:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 35, 372724), True, datetime.datetime(2023, 8, 8, 6, 4, 35, 372724), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:04:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:04:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:04:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:04:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:04:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:39.579576+00:00 (in 2.999203 seconds) namespacegcworker stdout | 2023-08-08 06:04:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:36 UTC)" (scheduled at 2023-08-08 06:04:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:04:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:04:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:04:37,179 [198] [DEBUG] [app] Starting request: urn:request:4eb3ba7f-c6b3-4b28-8c2e-641fd3eb27b4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:04:37,179 [200] [DEBUG] [app] Starting request: urn:request:568bba3b-1d83-4c84-a1ca-2398de465ac5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:04:37,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:37,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:04:37,185 [205] [DEBUG] [app] Starting request: urn:request:0c85db4f-3106-4f55-ab73-007df8dab813 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:04:37,186 [205] [DEBUG] [app] Ending request: urn:request:0c85db4f-3106-4f55-ab73-007df8dab813 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0c85db4f-3106-4f55-ab73-007df8dab813', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:04:37,186 [210] [DEBUG] [app] Starting request: urn:request:ed91f95b-43b9-43fe-bac6-56b69f2709fa (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:04:37,186 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:04:37,186 [210] [DEBUG] [app] Ending request: urn:request:ed91f95b-43b9-43fe-bac6-56b69f2709fa (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ed91f95b-43b9-43fe-bac6-56b69f2709fa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:04:37,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:04:37,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:37,189 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:37,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:37,192 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:37,193 [200] [DEBUG] [app] Starting request: urn:request:3f6f3991-9811-4bab-a99c-88778baff2ad (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:04:37,194 [200] [DEBUG] [app] Ending request: urn:request:3f6f3991-9811-4bab-a99c-88778baff2ad (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3f6f3991-9811-4bab-a99c-88778baff2ad', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:04:37,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:37,194 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:37,195 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:04:37,195 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:37,196 [197] [DEBUG] [app] Starting request: urn:request:3c999e18-db14-4796-9dd3-59def416b6c8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:04:37,197 [197] [DEBUG] [app] Ending request: urn:request:3c999e18-db14-4796-9dd3-59def416b6c8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3c999e18-db14-4796-9dd3-59def416b6c8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:04:37,197 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" buildlogsarchiver stdout | 2023-08-08 06:04:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:04:37,197 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 buildlogsarchiver stdout | 2023-08-08 06:04:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:07.197412+00:00 (in 29.999483 seconds) buildlogsarchiver stdout | 2023-08-08 06:04:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:07 UTC)" (scheduled at 2023-08-08 06:04:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:04:37,198 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:04:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 4, 37, 198349), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:04:37,200 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:04:37,204 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:04:37,206 [198] [DEBUG] [app] Ending request: urn:request:4eb3ba7f-c6b3-4b28-8c2e-641fd3eb27b4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4eb3ba7f-c6b3-4b28-8c2e-641fd3eb27b4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 06:04:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:04:37,206 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:04:37,206 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:04:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:04:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.029) buildlogsarchiver stdout | 2023-08-08 06:04:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:04:37,208 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:04:37,210 [200] [DEBUG] [app] Ending request: urn:request:568bba3b-1d83-4c84-a1ca-2398de465ac5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:568bba3b-1d83-4c84-a1ca-2398de465ac5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:04:37,210 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:04:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.032 47 0.033) gunicorn-web stdout | 2023-08-08 06:04:37,210 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:04:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:04:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:04:37,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:04:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:04:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:49.417660+00:00 (in 9.999462 seconds) gcworker stdout | 2023-08-08 06:04:39,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:49 UTC)" (scheduled at 2023-08-08 06:04:39.417660+00:00) gcworker stdout | 2023-08-08 06:04:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265079419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:04:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:04:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:04:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:04:39,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:04:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:54.393627+00:00 (in 14.813433 seconds) namespacegcworker stdout | 2023-08-08 06:04:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:09:39 UTC)" (scheduled at 2023-08-08 06:04:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:04:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 4, 39, 580651), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:04:39,588 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 4, 39, 580651), True, datetime.datetime(2023, 8, 8, 6, 4, 39, 580651), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:04:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 4, 39, 580651), True, datetime.datetime(2023, 8, 8, 6, 4, 39, 580651), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 4, 39, 580651), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:04:39,591 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:04:39,591 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:09:39 UTC)" executed successfully builder stdout | 2023-08-08 06:04:41,831 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:04:41,841 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:04:41,841 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:04:41,846 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:04:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:46.874674+00:00 (in 4.580346 seconds) chunkcleanupworker stdout | 2023-08-08 06:04:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:52 UTC)" (scheduled at 2023-08-08 06:04:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:04:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:04:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 42, 294922), True, datetime.datetime(2023, 8, 8, 6, 4, 42, 294922), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:04:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:04:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:04:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:04:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:55.371682+00:00 (in 9.999472 seconds) notificationworker stdout | 2023-08-08 06:04:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:55 UTC)" (scheduled at 2023-08-08 06:04:45.371682+00:00) notificationworker stdout | 2023-08-08 06:04:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:04:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 45, 372684), True, datetime.datetime(2023, 8, 8, 6, 4, 45, 372684), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:04:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:04:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:04:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:04:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:04:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:52.293757+00:00 (in 5.418551 seconds) chunkcleanupworker stdout | 2023-08-08 06:04:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:09:46 UTC)" (scheduled at 2023-08-08 06:04:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:04:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 4, 46, 875658), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:04:46,882 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 4, 46, 875658), True, datetime.datetime(2023, 8, 8, 6, 4, 46, 875658), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:04:46,884 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 4, 46, 875658), True, datetime.datetime(2023, 8, 8, 6, 4, 46, 875658), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 4, 46, 875658), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:04:46,885 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:09:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:04:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:04:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:25.514341+00:00 (in 38.114691 seconds) exportactionlogsworker stdout | 2023-08-08 06:04:47,400 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:47 UTC)" (scheduled at 2023-08-08 06:04:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:04:47,400 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:04:47,401 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 47, 400499), True, datetime.datetime(2023, 8, 8, 6, 4, 47, 400499), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:04:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:04:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:04:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:04:47,730 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:04:48,945 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:04:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:04:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:59.417660+00:00 (in 9.999474 seconds) gcworker stdout | 2023-08-08 06:04:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:59 UTC)" (scheduled at 2023-08-08 06:04:49.417660+00:00) gcworker stdout | 2023-08-08 06:04:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265089418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:04:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:04:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:04:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:04:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:04:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:04:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:04:56.890741+00:00 (in 7.202896 seconds) securityworker stdout | 2023-08-08 06:04:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:19 UTC)" (scheduled at 2023-08-08 06:04:49.687363+00:00) securityworker stdout | 2023-08-08 06:04:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:04:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:04:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:04:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:04:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:04:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:04:50,068 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:04:50,505 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:04:51,851 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:04:51,860 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:04:51,860 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:04:51,865 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:04:52,179 [197] [DEBUG] [app] Starting request: urn:request:9ccc7bc6-cab2-4f96-b153-358bff445c63 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:04:52,180 [200] [DEBUG] [app] Starting request: urn:request:3e7de1fb-b8eb-472a-8d86-f07ab6774389 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:04:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:52,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:04:52,185 [203] [DEBUG] [app] Starting request: urn:request:86b3ad34-a19e-4e87-973f-2036b550cc81 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:04:52,186 [210] [DEBUG] [app] Starting request: urn:request:f62cca80-c893-4bc1-8b75-cf5b28ad19b5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:04:52,186 [203] [DEBUG] [app] Ending request: urn:request:86b3ad34-a19e-4e87-973f-2036b550cc81 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:86b3ad34-a19e-4e87-973f-2036b550cc81', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:04:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:04:52,186 [210] [DEBUG] [app] Ending request: urn:request:f62cca80-c893-4bc1-8b75-cf5b28ad19b5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f62cca80-c893-4bc1-8b75-cf5b28ad19b5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:04:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:04:52,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:04:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:04:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:52,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:04:52,191 [198] [DEBUG] [app] Starting request: urn:request:6051be69-ea70-47b1-a39c-42a84e9a569f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:04:52,192 [199] [DEBUG] [app] Starting request: urn:request:c0aad622-2c69-4a93-9d28-695cb9e93c47 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:04:52,192 [198] [DEBUG] [app] Ending request: urn:request:6051be69-ea70-47b1-a39c-42a84e9a569f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6051be69-ea70-47b1-a39c-42a84e9a569f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:04:52,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:52,192 [199] [DEBUG] [app] Ending request: urn:request:c0aad622-2c69-4a93-9d28-695cb9e93c47 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c0aad622-2c69-4a93-9d28-695cb9e93c47', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:04:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:04:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:04:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:04:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:04:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:04:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:04:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:04:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:04:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:04:52,204 [200] [DEBUG] [app] Ending request: urn:request:3e7de1fb-b8eb-472a-8d86-f07ab6774389 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3e7de1fb-b8eb-472a-8d86-f07ab6774389', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:04:52,204 [197] [DEBUG] [app] Ending request: urn:request:9ccc7bc6-cab2-4f96-b153-358bff445c63 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9ccc7bc6-cab2-4f96-b153-358bff445c63', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:04:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:04:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:04:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:04:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:04:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:04:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:04:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:04:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:04:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:04:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:02.293757+00:00 (in 9.999495 seconds) chunkcleanupworker stdout | 2023-08-08 06:04:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:02 UTC)" (scheduled at 2023-08-08 06:04:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:04:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:04:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 52, 294631), True, datetime.datetime(2023, 8, 8, 6, 4, 52, 294631), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:04:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:04:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:04:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:04:53,246 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:04:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:04:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:13.097375+00:00 (in 18.812736 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:04:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:54 UTC)" (scheduled at 2023-08-08 06:04:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:04:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:04:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:04:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:04:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:36.579841+00:00 (in 42.185711 seconds) namespacegcworker stdout | 2023-08-08 06:04:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:54 UTC)" (scheduled at 2023-08-08 06:04:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:04:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:04:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 54, 394583), True, datetime.datetime(2023, 8, 8, 6, 4, 54, 394583), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:04:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:04:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:04:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:05:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:04:54,927 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:04:54,931 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:04:54,935 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:04:55,110 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:04:55,226 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:04:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:04:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:02.178530+00:00 (in 6.806300 seconds) notificationworker stdout | 2023-08-08 06:04:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:05 UTC)" (scheduled at 2023-08-08 06:04:55.371682+00:00) notificationworker stdout | 2023-08-08 06:04:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:04:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 4, 55, 372735), True, datetime.datetime(2023, 8, 8, 6, 4, 55, 372735), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:04:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:04:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:04:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:05 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:04:55,465 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:04:56,557 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:04:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:04:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:19.687363+00:00 (in 22.796086 seconds) securityworker stdout | 2023-08-08 06:04:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:26 UTC)" (scheduled at 2023-08-08 06:04:56.890741+00:00) securityworker stdout | 2023-08-08 06:04:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:04:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:04:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:04:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:04:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:04:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:04:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:04:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:04:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:04:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:04:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 5, 59, 56, 896532), 1, 2]) securityworker stdout | 2023-08-08 06:04:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 5, 59, 56, 896532), 1, 2]) securityworker stdout | 2023-08-08 06:04:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:04:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:04:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:04:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:04:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:04:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:04:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:04:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:04:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:04:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:04:57,102 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:04:57,192 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:04:57,335 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:04:57,393 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:04:57,415 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:04:57,434 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:04:57,437 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:04:58,535 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:04:58,689 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:04:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:04:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:09.417660+00:00 (in 9.999464 seconds) gcworker stdout | 2023-08-08 06:04:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:09 UTC)" (scheduled at 2023-08-08 06:04:59.417660+00:00) gcworker stdout | 2023-08-08 06:04:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265099418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:04:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:04:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:04:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:04:59,775 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:05:00,606 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:05:00,699 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:05:01,101 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:05:01,666 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} builder stdout | 2023-08-08 06:05:01,868 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:05:01,877 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:05:01,877 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:05:01,881 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:05:01,986 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} notificationworker stdout | 2023-08-08 06:05:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:05:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:05.371682+00:00 (in 3.192564 seconds) notificationworker stdout | 2023-08-08 06:05:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:02 UTC)" (scheduled at 2023-08-08 06:05:02.178530+00:00) notificationworker stdout | 2023-08-08 06:05:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:05:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:05:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:05:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:12.293757+00:00 (in 9.999506 seconds) chunkcleanupworker stdout | 2023-08-08 06:05:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:12 UTC)" (scheduled at 2023-08-08 06:05:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:05:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:05:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 2, 294599), True, datetime.datetime(2023, 8, 8, 6, 5, 2, 294599), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:05:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:05:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:05:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:12 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:05:02,318 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:05:03,180 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:05:03,183 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:05:03,191 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:05:03,194 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:05:03,197 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:05:03,202 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:05:03,204 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:05:03,207 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:05:03,210 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:05:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:05:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:09.793658+00:00 (in 5.605718 seconds) repositorygcworker stdout | 2023-08-08 06:05:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:04 UTC)" (scheduled at 2023-08-08 06:05:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:05:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:05:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:05:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:05:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:15.371682+00:00 (in 9.999011 seconds) notificationworker stdout | 2023-08-08 06:05:05,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:15 UTC)" (scheduled at 2023-08-08 06:05:05.371682+00:00) notificationworker stdout | 2023-08-08 06:05:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:05:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 5, 373167), True, datetime.datetime(2023, 8, 8, 6, 5, 5, 373167), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:05:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:05:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:05:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:05:07,181 [197] [DEBUG] [app] Starting request: urn:request:89cf2699-aa9c-4b89-81a3-7401ad8c84a9 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:05:07,181 [198] [DEBUG] [app] Starting request: urn:request:40a586c7-063a-4707-9427-db12c1ab878e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:05:07,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:07,182 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:07,185 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:07,186 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:05:07,186 [210] [DEBUG] [app] Starting request: urn:request:18062b6f-80a9-42a6-aada-c68fca8d2d3b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:05:07,187 [210] [DEBUG] [app] Ending request: urn:request:18062b6f-80a9-42a6-aada-c68fca8d2d3b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:18062b6f-80a9-42a6-aada-c68fca8d2d3b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:05:07,187 [209] [DEBUG] [app] Starting request: urn:request:b5c2fc4b-5408-4ad3-9628-ec4375f5b9d9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:05:07,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:05:07,188 [209] [DEBUG] [app] Ending request: urn:request:b5c2fc4b-5408-4ad3-9628-ec4375f5b9d9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b5c2fc4b-5408-4ad3-9628-ec4375f5b9d9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:05:07,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:05:07,188 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:05:07,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:07,189 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:07,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:07,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:07,192 [197] [DEBUG] [app] Starting request: urn:request:1ad6b908-26f9-4991-ad45-cd2f4588cdcf (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:05:07,193 [197] [DEBUG] [app] Ending request: urn:request:1ad6b908-26f9-4991-ad45-cd2f4588cdcf (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1ad6b908-26f9-4991-ad45-cd2f4588cdcf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:05:07,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:05:07,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:07,194 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:05:07,194 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:07,195 [199] [DEBUG] [app] Starting request: urn:request:e69659e7-501f-435f-8fab-8276ca2d1f80 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:05:07,195 [199] [DEBUG] [app] Ending request: urn:request:e69659e7-501f-435f-8fab-8276ca2d1f80 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e69659e7-501f-435f-8fab-8276ca2d1f80', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2023-08-08 06:05:07,196 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:05:07,196 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:07,196 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:05:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:05:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:37.197412+00:00 (in 29.999536 seconds) buildlogsarchiver stdout | 2023-08-08 06:05:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:37 UTC)" (scheduled at 2023-08-08 06:05:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:05:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 5, 7, 198304), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:05:07,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:05:07,201 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:05:07,205 [198] [DEBUG] [app] Ending request: urn:request:40a586c7-063a-4707-9427-db12c1ab878e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:40a586c7-063a-4707-9427-db12c1ab878e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 06:05:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:05:07,205 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:07,206 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:05:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:05:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:05:07,206 [197] [DEBUG] [app] Ending request: urn:request:89cf2699-aa9c-4b89-81a3-7401ad8c84a9 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:89cf2699-aa9c-4b89-81a3-7401ad8c84a9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 06:05:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 gunicorn-web stdout | 2023-08-08 06:05:07,206 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:05:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:05:07,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:05:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:05:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:05:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:05:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:05:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:05:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:19.417660+00:00 (in 9.999512 seconds) gcworker stdout | 2023-08-08 06:05:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:19 UTC)" (scheduled at 2023-08-08 06:05:09.417660+00:00) gcworker stdout | 2023-08-08 06:05:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265109418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:05:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:05:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:05:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:05:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:05:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:04.187429+00:00 (in 54.393229 seconds) repositorygcworker stdout | 2023-08-08 06:05:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:09 UTC)" (scheduled at 2023-08-08 06:05:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:05:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:05:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 9, 795136), True, datetime.datetime(2023, 8, 8, 6, 5, 9, 795136), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:05:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:05:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:05:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:09 UTC)" executed successfully builder stdout | 2023-08-08 06:05:11,882 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:05:11,892 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:05:11,893 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:05:11,897 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:05:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:05:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:22.293757+00:00 (in 9.999500 seconds) chunkcleanupworker stdout | 2023-08-08 06:05:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:22 UTC)" (scheduled at 2023-08-08 06:05:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:05:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:05:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 12, 294603), True, datetime.datetime(2023, 8, 8, 6, 5, 12, 294603), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:05:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:05:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:05:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:05:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:05:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:54.284093+00:00 (in 41.186155 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:05:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:13 UTC)" (scheduled at 2023-08-08 06:05:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:05:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:05:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 13, 98403), True, datetime.datetime(2023, 8, 8, 6, 5, 13, 98403), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:05:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:05:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:05:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:05:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:05:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:25.371682+00:00 (in 9.999450 seconds) notificationworker stdout | 2023-08-08 06:05:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:25 UTC)" (scheduled at 2023-08-08 06:05:15.371682+00:00) notificationworker stdout | 2023-08-08 06:05:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:05:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 15, 372753), True, datetime.datetime(2023, 8, 8, 6, 5, 15, 372753), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:05:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:05:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:05:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:05:17,768 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:05:18,983 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:05:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:05:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:29.417660+00:00 (in 9.999509 seconds) gcworker stdout | 2023-08-08 06:05:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:29 UTC)" (scheduled at 2023-08-08 06:05:19.417660+00:00) gcworker stdout | 2023-08-08 06:05:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265119418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:05:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:05:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:05:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:05:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:05:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:26.890741+00:00 (in 7.202845 seconds) securityworker stdout | 2023-08-08 06:05:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:49 UTC)" (scheduled at 2023-08-08 06:05:19.687363+00:00) securityworker stdout | 2023-08-08 06:05:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:05:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:05:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:05:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:05:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:05:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:05:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 0, 19, 692093), 1, 2]) securityworker stdout | 2023-08-08 06:05:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 0, 19, 692093), 1, 2]) securityworker stdout | 2023-08-08 06:05:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:05:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:05:20,077 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:05:20,541 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:05:21,903 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:05:21,914 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:05:21,914 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:05:21,919 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:22,179 [197] [DEBUG] [app] Starting request: urn:request:deee5ecd-bfd1-4e37-9b2d-94614d5f13dc (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:05:22,179 [199] [DEBUG] [app] Starting request: urn:request:a1f01056-c386-4e2b-aa46-eacb4093ceac (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:05:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:05:22,185 [203] [DEBUG] [app] Starting request: urn:request:4bc7e434-f691-4250-a9e8-2a6a5cf7c670 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:05:22,185 [203] [DEBUG] [app] Ending request: urn:request:4bc7e434-f691-4250-a9e8-2a6a5cf7c670 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4bc7e434-f691-4250-a9e8-2a6a5cf7c670', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:05:22,186 [210] [DEBUG] [app] Starting request: urn:request:db54920e-74a4-4f6e-8ec8-68cd2ebc4edd (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:05:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:05:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:05:22,186 [210] [DEBUG] [app] Ending request: urn:request:db54920e-74a4-4f6e-8ec8-68cd2ebc4edd (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:db54920e-74a4-4f6e-8ec8-68cd2ebc4edd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:05:22,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:05:22,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:22,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:22,191 [199] [DEBUG] [app] Starting request: urn:request:a32881d5-1edf-4278-b205-39f53bbc7a17 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:05:22,192 [199] [DEBUG] [app] Ending request: urn:request:a32881d5-1edf-4278-b205-39f53bbc7a17 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a32881d5-1edf-4278-b205-39f53bbc7a17', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:05:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:05:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:22,193 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:05:22,194 [200] [DEBUG] [app] Starting request: urn:request:f03ef03b-2e8c-4472-a0fe-5899feb33612 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:05:22,195 [200] [DEBUG] [app] Ending request: urn:request:f03ef03b-2e8c-4472-a0fe-5899feb33612 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f03ef03b-2e8c-4472-a0fe-5899feb33612', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:05:22,195 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:05:22,195 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:22,196 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:05:22,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:05:22,200 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:05:22,204 [197] [DEBUG] [app] Ending request: urn:request:deee5ecd-bfd1-4e37-9b2d-94614d5f13dc (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:deee5ecd-bfd1-4e37-9b2d-94614d5f13dc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:05:22,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:05:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:05:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:05:22,206 [199] [DEBUG] [app] Ending request: urn:request:a1f01056-c386-4e2b-aa46-eacb4093ceac (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a1f01056-c386-4e2b-aa46-eacb4093ceac', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:05:22,206 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:22,206 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:05:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:05:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:05:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:05:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:28.874906+00:00 (in 6.580649 seconds) chunkcleanupworker stdout | 2023-08-08 06:05:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:32 UTC)" (scheduled at 2023-08-08 06:05:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:05:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:05:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 22, 294631), True, datetime.datetime(2023, 8, 8, 6, 5, 22, 294631), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:05:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:05:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:05:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:05:23,284 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:05:24,936 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:05:24,941 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:05:24,944 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:05:25,117 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:05:25,262 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:05:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:05:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:35.371682+00:00 (in 9.999416 seconds) notificationworker stdout | 2023-08-08 06:05:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:35 UTC)" (scheduled at 2023-08-08 06:05:25.371682+00:00) notificationworker stdout | 2023-08-08 06:05:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:05:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 25, 372793), True, datetime.datetime(2023, 8, 8, 6, 5, 25, 372793), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:05:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:05:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:05:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:35 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:05:25,503 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} exportactionlogsworker stdout | 2023-08-08 06:05:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:05:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:47.398551+00:00 (in 21.883595 seconds) exportactionlogsworker stdout | 2023-08-08 06:05:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:25 UTC)" (scheduled at 2023-08-08 06:05:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:05:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:05:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:25 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:05:26,595 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:05:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:05:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:49.687363+00:00 (in 22.796119 seconds) securityworker stdout | 2023-08-08 06:05:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:56 UTC)" (scheduled at 2023-08-08 06:05:26.890741+00:00) securityworker stdout | 2023-08-08 06:05:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:05:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:05:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:05:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:05:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:05:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:05:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:05:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:05:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:05:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:05:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 0, 26, 896429), 1, 2]) securityworker stdout | 2023-08-08 06:05:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 0, 26, 896429), 1, 2]) securityworker stdout | 2023-08-08 06:05:26,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:26,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:26,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:05:26,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:05:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:05:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:05:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:05:26,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:05:26,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:05:26,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:05:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:05:27,130 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:05:27,223 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:05:27,344 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:05:27,406 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:05:27,429 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:05:27,449 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:05:27,452 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:05:28,554 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:05:28,699 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 06:05:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:05:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:32.293757+00:00 (in 3.418281 seconds) chunkcleanupworker stdout | 2023-08-08 06:05:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:28 UTC)" (scheduled at 2023-08-08 06:05:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:05:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:05:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:05:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:05:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:39.417660+00:00 (in 9.999364 seconds) gcworker stdout | 2023-08-08 06:05:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:39 UTC)" (scheduled at 2023-08-08 06:05:29.417660+00:00) gcworker stdout | 2023-08-08 06:05:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265129418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:05:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:05:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:05:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:05:29,812 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:05:30,644 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:05:30,738 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:05:31,139 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:05:31,690 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} builder stdout | 2023-08-08 06:05:31,929 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:05:31,939 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:05:31,939 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:05:31,943 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:05:32,023 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} chunkcleanupworker stdout | 2023-08-08 06:05:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:05:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:42.293757+00:00 (in 9.999405 seconds) chunkcleanupworker stdout | 2023-08-08 06:05:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:42 UTC)" (scheduled at 2023-08-08 06:05:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:05:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:05:32,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 32, 294874), True, datetime.datetime(2023, 8, 8, 6, 5, 32, 294874), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:05:32,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:05:32,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:05:32,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:42 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:05:32,329 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:05:33,188 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:05:33,192 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:05:33,200 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:05:33,207 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:05:33,210 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:05:33,212 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:05:33,217 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:05:33,220 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:05:33,223 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:05:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:05:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:45.371682+00:00 (in 9.999424 seconds) notificationworker stdout | 2023-08-08 06:05:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:45 UTC)" (scheduled at 2023-08-08 06:05:35.371682+00:00) notificationworker stdout | 2023-08-08 06:05:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:05:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 35, 372681), True, datetime.datetime(2023, 8, 8, 6, 5, 35, 372681), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:05:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:05:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:05:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:05:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:05:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:54.393627+00:00 (in 17.813248 seconds) namespacegcworker stdout | 2023-08-08 06:05:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:36 UTC)" (scheduled at 2023-08-08 06:05:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:05:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:05:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:05:37,179 [197] [DEBUG] [app] Starting request: urn:request:35374b95-799a-4d42-86dd-746787431fa3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:05:37,180 [200] [DEBUG] [app] Starting request: urn:request:927c43ed-50e8-4208-b7d3-ff167f341ed3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:05:37,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:37,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:37,186 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:05:37,187 [207] [DEBUG] [app] Starting request: urn:request:5ef30a46-fa80-49ea-a900-ba921ebbbc72 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:05:37,187 [207] [DEBUG] [app] Ending request: urn:request:5ef30a46-fa80-49ea-a900-ba921ebbbc72 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5ef30a46-fa80-49ea-a900-ba921ebbbc72', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:05:37,187 [203] [DEBUG] [app] Starting request: urn:request:fece8a62-5237-4d58-bf13-38bb1f7e78a8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:05:37,188 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:05:37,188 [203] [DEBUG] [app] Ending request: urn:request:fece8a62-5237-4d58-bf13-38bb1f7e78a8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fece8a62-5237-4d58-bf13-38bb1f7e78a8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:05:37,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:05:37,188 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:05:37,189 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:37,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:37,190 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:37,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:37,193 [200] [DEBUG] [app] Starting request: urn:request:c3d3299b-7ef5-4c24-ac79-1c48706744cf (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:05:37,194 [200] [DEBUG] [app] Ending request: urn:request:c3d3299b-7ef5-4c24-ac79-1c48706744cf (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c3d3299b-7ef5-4c24-ac79-1c48706744cf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:05:37,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:05:37,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:37,195 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:37,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:05:37,196 [199] [DEBUG] [app] Starting request: urn:request:a67b294c-7a21-45c3-83ec-e98b793bff90 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:05:37,197 [199] [DEBUG] [app] Ending request: urn:request:a67b294c-7a21-45c3-83ec-e98b793bff90 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a67b294c-7a21-45c3-83ec-e98b793bff90', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:05:37,197 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" buildlogsarchiver stdout | 2023-08-08 06:05:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:05:37,197 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 buildlogsarchiver stdout | 2023-08-08 06:05:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:07.197412+00:00 (in 29.999506 seconds) gunicorn-web stdout | 2023-08-08 06:05:37,198 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:05:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:07 UTC)" (scheduled at 2023-08-08 06:05:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:05:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 5, 37, 198792), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:05:37,200 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:05:37,202 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:05:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:05:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 gunicorn-web stdout | 2023-08-08 06:05:37,208 [197] [DEBUG] [app] Ending request: urn:request:35374b95-799a-4d42-86dd-746787431fa3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:35374b95-799a-4d42-86dd-746787431fa3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 06:05:37,208 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:37,208 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:37,208 [200] [DEBUG] [app] Ending request: urn:request:927c43ed-50e8-4208-b7d3-ff167f341ed3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:927c43ed-50e8-4208-b7d3-ff167f341ed3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:05:37,208 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:37,208 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:05:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:05:37,209 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:05:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:05:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.031 47 0.030) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:05:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.031 47 0.031) buildlogsarchiver stdout | 2023-08-08 06:05:37,212 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:05:37,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:05:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:05:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:49.417660+00:00 (in 9.999483 seconds) gcworker stdout | 2023-08-08 06:05:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:49 UTC)" (scheduled at 2023-08-08 06:05:39.417660+00:00) gcworker stdout | 2023-08-08 06:05:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265139418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:05:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:05:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:05:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:49 UTC)" executed successfully builder stdout | 2023-08-08 06:05:41,950 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:05:41,960 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:05:41,960 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:05:41,965 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:05:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:05:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:52.293757+00:00 (in 9.999490 seconds) chunkcleanupworker stdout | 2023-08-08 06:05:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:52 UTC)" (scheduled at 2023-08-08 06:05:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:05:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:05:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 42, 294626), True, datetime.datetime(2023, 8, 8, 6, 5, 42, 294626), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:05:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:05:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:05:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:05:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:05:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:55.371682+00:00 (in 9.999423 seconds) notificationworker stdout | 2023-08-08 06:05:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:55 UTC)" (scheduled at 2023-08-08 06:05:45.371682+00:00) notificationworker stdout | 2023-08-08 06:05:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:05:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 45, 372647), True, datetime.datetime(2023, 8, 8, 6, 5, 45, 372647), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:05:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:05:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:05:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:05:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:05:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:25.514341+00:00 (in 38.115263 seconds) exportactionlogsworker stdout | 2023-08-08 06:05:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:47 UTC)" (scheduled at 2023-08-08 06:05:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:05:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:05:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 47, 399634), True, datetime.datetime(2023, 8, 8, 6, 5, 47, 399634), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:05:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:05:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:05:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:05:47,805 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:05:49,020 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:05:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:05:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:59.417660+00:00 (in 9.999501 seconds) gcworker stdout | 2023-08-08 06:05:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:59 UTC)" (scheduled at 2023-08-08 06:05:49.417660+00:00) gcworker stdout | 2023-08-08 06:05:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265149418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:05:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:05:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:05:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:05:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:05:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:05:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:05:56.890741+00:00 (in 7.202840 seconds) securityworker stdout | 2023-08-08 06:05:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:19 UTC)" (scheduled at 2023-08-08 06:05:49.687363+00:00) securityworker stdout | 2023-08-08 06:05:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:05:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:05:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:05:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:05:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:05:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:05:50,101 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:05:50,554 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:05:51,969 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:05:51,979 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:05:51,979 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:05:51,984 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:52,178 [198] [DEBUG] [app] Starting request: urn:request:cfd20016-c836-4602-8c62-71e90083907a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:05:52,179 [199] [DEBUG] [app] Starting request: urn:request:441edabf-7437-40bf-af70-0bb0fca7e1a2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:05:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:52,182 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:05:52,184 [205] [DEBUG] [app] Starting request: urn:request:25666168-d003-4674-9933-22e6d515616f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:05:52,184 [205] [DEBUG] [app] Ending request: urn:request:25666168-d003-4674-9933-22e6d515616f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:25666168-d003-4674-9933-22e6d515616f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:05:52,184 [203] [DEBUG] [app] Starting request: urn:request:f7809148-c389-46bd-9e7e-96de245ddaeb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:05:52,185 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:05:52,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:05:52,185 [203] [DEBUG] [app] Ending request: urn:request:f7809148-c389-46bd-9e7e-96de245ddaeb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f7809148-c389-46bd-9e7e-96de245ddaeb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:05:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:05:52,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:52,186 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:52,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:05:52,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:52,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:05:52,190 [197] [DEBUG] [app] Starting request: urn:request:9b8a05a7-288c-43c4-bf9a-de9f46760e27 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:05:52,190 [197] [DEBUG] [app] Ending request: urn:request:9b8a05a7-288c-43c4-bf9a-de9f46760e27 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9b8a05a7-288c-43c4-bf9a-de9f46760e27', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:05:52,190 [200] [DEBUG] [app] Starting request: urn:request:4d996dd9-d4bc-419f-a958-58e7249f1331 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:05:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:05:52,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:52,191 [200] [DEBUG] [app] Ending request: urn:request:4d996dd9-d4bc-419f-a958-58e7249f1331 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4d996dd9-d4bc-419f-a958-58e7249f1331', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:05:52,191 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:05:52,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:05:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:05:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:05:52,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:05:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:05:52,196 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:05:52,196 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:05:52,202 [198] [DEBUG] [app] Ending request: urn:request:cfd20016-c836-4602-8c62-71e90083907a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cfd20016-c836-4602-8c62-71e90083907a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:05:52,202 [199] [DEBUG] [app] Ending request: urn:request:441edabf-7437-40bf-af70-0bb0fca7e1a2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:441edabf-7437-40bf-af70-0bb0fca7e1a2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:05:52,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:52,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:05:52,202 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:05:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:05:52,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:05:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:05:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:05:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) chunkcleanupworker stdout | 2023-08-08 06:05:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:05:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:02.293757+00:00 (in 9.999528 seconds) chunkcleanupworker stdout | 2023-08-08 06:05:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:02 UTC)" (scheduled at 2023-08-08 06:05:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:05:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:05:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 52, 294585), True, datetime.datetime(2023, 8, 8, 6, 5, 52, 294585), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:05:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:05:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:05:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:05:53,298 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:05:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:05:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:13.097375+00:00 (in 18.812681 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:05:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:54 UTC)" (scheduled at 2023-08-08 06:05:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:05:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:05:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:05:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:05:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:36.579841+00:00 (in 42.185655 seconds) namespacegcworker stdout | 2023-08-08 06:05:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:54 UTC)" (scheduled at 2023-08-08 06:05:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:05:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:05:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 54, 394545), True, datetime.datetime(2023, 8, 8, 6, 5, 54, 394545), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:05:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:05:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:05:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:06:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:05:54,945 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:05:54,949 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:05:54,954 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:05:55,154 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:05:55,300 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:05:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:05:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:02.178530+00:00 (in 6.806345 seconds) notificationworker stdout | 2023-08-08 06:05:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:05 UTC)" (scheduled at 2023-08-08 06:05:55.371682+00:00) notificationworker stdout | 2023-08-08 06:05:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:05:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 5, 55, 372577), True, datetime.datetime(2023, 8, 8, 6, 5, 55, 372577), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:05:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:05:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:05:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:05 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:05:55,510 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:05:56,631 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:05:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:05:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:19.687363+00:00 (in 22.796109 seconds) securityworker stdout | 2023-08-08 06:05:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:26 UTC)" (scheduled at 2023-08-08 06:05:56.890741+00:00) securityworker stdout | 2023-08-08 06:05:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:05:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:05:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:05:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:05:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:05:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:05:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:05:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:05:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:05:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:05:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 0, 56, 896510), 1, 2]) securityworker stdout | 2023-08-08 06:05:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 0, 56, 896510), 1, 2]) securityworker stdout | 2023-08-08 06:05:56,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:05:56,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:05:56,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:05:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:05:56,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:05:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:05:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:05:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:05:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:05:56,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:05:57,167 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:05:57,259 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:05:57,355 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:05:57,419 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:05:57,444 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:05:57,464 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:05:57,467 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} buildlogsarchiver stdout | 2023-08-08 06:05:58,578 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:05:58,737 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:05:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:05:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:09.417660+00:00 (in 9.999499 seconds) gcworker stdout | 2023-08-08 06:05:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:09 UTC)" (scheduled at 2023-08-08 06:05:59.417660+00:00) gcworker stdout | 2023-08-08 06:05:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265159418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:05:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:05:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:05:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:05:59,839 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:06:00,666 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:06:00,769 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:06:01,174 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:06:01,723 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} builder stdout | 2023-08-08 06:06:01,994 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:06:02,003 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:06:02,003 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:06:02,008 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:06:02,033 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} notificationworker stdout | 2023-08-08 06:06:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:06:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:05.371682+00:00 (in 3.192636 seconds) notificationworker stdout | 2023-08-08 06:06:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:02 UTC)" (scheduled at 2023-08-08 06:06:02.178530+00:00) notificationworker stdout | 2023-08-08 06:06:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:06:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:06:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:06:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:12.293757+00:00 (in 9.999511 seconds) chunkcleanupworker stdout | 2023-08-08 06:06:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:12 UTC)" (scheduled at 2023-08-08 06:06:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:06:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:06:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 2, 294609), True, datetime.datetime(2023, 8, 8, 6, 6, 2, 294609), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:06:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:06:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:12 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:06:02,368 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:06:03,202 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:06:03,206 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:06:03,209 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:06:03,217 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:06:03,220 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:06:03,225 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:06:03,228 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:06:03,231 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:06:03,234 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:06:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:06:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:09.793658+00:00 (in 5.605629 seconds) repositorygcworker stdout | 2023-08-08 06:06:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:04 UTC)" (scheduled at 2023-08-08 06:06:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:06:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:06:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:06:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:06:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:15.371682+00:00 (in 9.999440 seconds) notificationworker stdout | 2023-08-08 06:06:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:15 UTC)" (scheduled at 2023-08-08 06:06:05.371682+00:00) notificationworker stdout | 2023-08-08 06:06:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:06:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 5, 372592), True, datetime.datetime(2023, 8, 8, 6, 6, 5, 372592), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:06:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:06:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:06:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:06:07,181 [197] [DEBUG] [app] Starting request: urn:request:6e546668-1100-4006-8143-5c3649f10b07 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:06:07,182 [198] [DEBUG] [app] Starting request: urn:request:70296564-3ec2-49fe-8a7f-7abc90a04d41 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:06:07,183 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:07,184 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:07,186 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:06:07,187 [208] [DEBUG] [app] Starting request: urn:request:a0c730db-ba86-498c-9332-b125561d99f7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:06:07,187 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:06:07,188 [208] [DEBUG] [app] Ending request: urn:request:a0c730db-ba86-498c-9332-b125561d99f7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a0c730db-ba86-498c-9332-b125561d99f7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:06:07,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:06:07,188 [203] [DEBUG] [app] Starting request: urn:request:3e064861-40e1-4084-8205-7b5b6aadd77e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:06:07,188 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:06:07,189 [203] [DEBUG] [app] Ending request: urn:request:3e064861-40e1-4084-8205-7b5b6aadd77e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3e064861-40e1-4084-8205-7b5b6aadd77e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:06:07,189 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:06:07,190 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:07,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:07,192 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:07,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:07,194 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:07,194 [200] [DEBUG] [app] Starting request: urn:request:c7b9f154-4c19-40e0-ad2a-f8396afa9237 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:06:07,195 [200] [DEBUG] [app] Ending request: urn:request:c7b9f154-4c19-40e0-ad2a-f8396afa9237 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c7b9f154-4c19-40e0-ad2a-f8396afa9237', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:06:07,195 [199] [DEBUG] [app] Starting request: urn:request:faf74814-b8d9-4875-98a9-24f40e244d8c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:06:07,195 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:06:07,196 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:07,196 [199] [DEBUG] [app] Ending request: urn:request:faf74814-b8d9-4875-98a9-24f40e244d8c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:faf74814-b8d9-4875-98a9-24f40e244d8c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:06:07,196 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:06:07,196 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:06:07,196 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:07,197 [198] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:06:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:06:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:37.197412+00:00 (in 29.999517 seconds) buildlogsarchiver stdout | 2023-08-08 06:06:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:37 UTC)" (scheduled at 2023-08-08 06:06:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:06:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 6, 7, 198302), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:06:07,201 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:06:07,202 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:06:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:06:07,206 [197] [DEBUG] [app] Ending request: urn:request:6e546668-1100-4006-8143-5c3649f10b07 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6e546668-1100-4006-8143-5c3649f10b07', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:06:07,207 [197] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:06:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:06:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) gunicorn-web stdout | 2023-08-08 06:06:07,207 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:06:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:06:07,208 [198] [DEBUG] [app] Ending request: urn:request:70296564-3ec2-49fe-8a7f-7abc90a04d41 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:70296564-3ec2-49fe-8a7f-7abc90a04d41', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 06:06:07,208 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:07,208 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:07,208 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:06:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:06:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:06:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:06:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:06:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:06:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:19.417660+00:00 (in 9.999503 seconds) gcworker stdout | 2023-08-08 06:06:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:19 UTC)" (scheduled at 2023-08-08 06:06:09.417660+00:00) gcworker stdout | 2023-08-08 06:06:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265169418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:06:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:06:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:06:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:06:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:06:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:04.187429+00:00 (in 54.392932 seconds) repositorygcworker stdout | 2023-08-08 06:06:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:09 UTC)" (scheduled at 2023-08-08 06:06:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:06:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:06:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 9, 795104), True, datetime.datetime(2023, 8, 8, 6, 6, 9, 795104), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:06:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:06:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:06:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:09 UTC)" executed successfully builder stdout | 2023-08-08 06:06:12,018 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:06:12,027 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:06:12,027 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:06:12,032 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:06:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:22.293757+00:00 (in 9.999462 seconds) chunkcleanupworker stdout | 2023-08-08 06:06:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:22 UTC)" (scheduled at 2023-08-08 06:06:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:06:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:06:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 12, 294692), True, datetime.datetime(2023, 8, 8, 6, 6, 12, 294692), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:06:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:06:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:06:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:06:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:31.283850+00:00 (in 18.185877 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:06:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:13 UTC)" (scheduled at 2023-08-08 06:06:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:06:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:06:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 13, 98516), True, datetime.datetime(2023, 8, 8, 6, 6, 13, 98516), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:06:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:06:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:06:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:06:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:06:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:25.371682+00:00 (in 9.999496 seconds) notificationworker stdout | 2023-08-08 06:06:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:25 UTC)" (scheduled at 2023-08-08 06:06:15.371682+00:00) notificationworker stdout | 2023-08-08 06:06:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:06:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 15, 372534), True, datetime.datetime(2023, 8, 8, 6, 6, 15, 372534), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:06:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:06:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:06:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:06:17,845 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:06:19,045 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:06:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:06:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:29.417660+00:00 (in 9.999486 seconds) gcworker stdout | 2023-08-08 06:06:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:29 UTC)" (scheduled at 2023-08-08 06:06:19.417660+00:00) gcworker stdout | 2023-08-08 06:06:19,418 [61] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2023-08-08 06:06:19,426 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265179425, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:06:19,430 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:06:19,430 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:06:19,430 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:06:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:06:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:26.890741+00:00 (in 7.202881 seconds) securityworker stdout | 2023-08-08 06:06:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:49 UTC)" (scheduled at 2023-08-08 06:06:19.687363+00:00) securityworker stdout | 2023-08-08 06:06:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:06:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:06:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:06:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:06:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:06:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:06:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 1, 19, 692098), 1, 2]) securityworker stdout | 2023-08-08 06:06:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 1, 19, 692098), 1, 2]) securityworker stdout | 2023-08-08 06:06:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:06:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:06:20,115 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:06:20,592 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:06:22,039 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:06:22,049 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:06:22,049 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:06:22,053 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:22,180 [197] [DEBUG] [app] Starting request: urn:request:f4249971-c8c4-486d-bf7d-ef8abbf69a82 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:06:22,181 [200] [DEBUG] [app] Starting request: urn:request:831bc438-30b5-47a9-8e20-4ab490852eac (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:06:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:22,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:22,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:06:22,186 [203] [DEBUG] [app] Starting request: urn:request:b6ea7fec-d13c-470a-937e-dcf44efb1a12 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:06:22,186 [208] [DEBUG] [app] Starting request: urn:request:37456666-0319-40de-806d-efa323b076a5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:06:22,186 [203] [DEBUG] [app] Ending request: urn:request:b6ea7fec-d13c-470a-937e-dcf44efb1a12 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b6ea7fec-d13c-470a-937e-dcf44efb1a12', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:06:22,187 [208] [DEBUG] [app] Ending request: urn:request:37456666-0319-40de-806d-efa323b076a5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:37456666-0319-40de-806d-efa323b076a5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:06:22,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:06:22,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:06:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:22,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:22,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:22,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:22,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:22,192 [200] [DEBUG] [app] Starting request: urn:request:0d033e72-d2e2-4453-92ca-1b525a168d03 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:06:22,193 [200] [DEBUG] [app] Ending request: urn:request:0d033e72-d2e2-4453-92ca-1b525a168d03 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0d033e72-d2e2-4453-92ca-1b525a168d03', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:06:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:06:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:22,193 [198] [DEBUG] [app] Starting request: urn:request:c224a453-5790-4290-bcda-a6e7a0ed7fb7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:06:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:06:22,194 [198] [DEBUG] [app] Ending request: urn:request:c224a453-5790-4290-bcda-a6e7a0ed7fb7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c224a453-5790-4290-bcda-a6e7a0ed7fb7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:06:22,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:06:22,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:22,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:06:22,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:06:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:06:22,205 [200] [DEBUG] [app] Ending request: urn:request:831bc438-30b5-47a9-8e20-4ab490852eac (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:831bc438-30b5-47a9-8e20-4ab490852eac', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:06:22,205 [197] [DEBUG] [app] Ending request: urn:request:f4249971-c8c4-486d-bf7d-ef8abbf69a82 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f4249971-c8c4-486d-bf7d-ef8abbf69a82', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:06:22,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:22,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:22,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:06:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:06:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:06:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:06:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:06:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:06:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:06:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:28.874906+00:00 (in 6.580591 seconds) chunkcleanupworker stdout | 2023-08-08 06:06:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:32 UTC)" (scheduled at 2023-08-08 06:06:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:06:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:06:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 22, 294784), True, datetime.datetime(2023, 8, 8, 6, 6, 22, 294784), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:06:22,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:06:22,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:22,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:06:23,335 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:06:24,953 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:06:24,958 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:06:24,961 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:06:25,166 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:06:25,338 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:06:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:06:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:35.371682+00:00 (in 9.999449 seconds) notificationworker stdout | 2023-08-08 06:06:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:35 UTC)" (scheduled at 2023-08-08 06:06:25.371682+00:00) notificationworker stdout | 2023-08-08 06:06:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:06:25,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 25, 372791), True, datetime.datetime(2023, 8, 8, 6, 6, 25, 372791), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:06:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:06:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:06:25,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:06:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:06:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:47.398551+00:00 (in 21.883678 seconds) exportactionlogsworker stdout | 2023-08-08 06:06:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:25 UTC)" (scheduled at 2023-08-08 06:06:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:06:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:06:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:25 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:06:25,546 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:06:26,665 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:06:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:06:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:49.687363+00:00 (in 22.796090 seconds) securityworker stdout | 2023-08-08 06:06:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:56 UTC)" (scheduled at 2023-08-08 06:06:26.890741+00:00) securityworker stdout | 2023-08-08 06:06:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:06:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:06:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:06:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:06:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:06:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:06:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:06:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:06:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:06:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:06:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 1, 26, 897091), 1, 2]) securityworker stdout | 2023-08-08 06:06:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 1, 26, 897091), 1, 2]) securityworker stdout | 2023-08-08 06:06:26,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:26,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:26,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:06:26,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:06:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:06:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:06:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:06:26,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:06:26,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:06:26,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:06:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:06:27,177 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:06:27,294 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:06:27,363 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:06:27,429 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:06:27,456 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:06:27,478 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:06:27,481 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} buildlogsarchiver stdout | 2023-08-08 06:06:28,608 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:06:28,776 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 06:06:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:06:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:32.293757+00:00 (in 3.418249 seconds) chunkcleanupworker stdout | 2023-08-08 06:06:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:28 UTC)" (scheduled at 2023-08-08 06:06:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:06:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:06:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:06:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:06:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:39.417660+00:00 (in 9.999494 seconds) gcworker stdout | 2023-08-08 06:06:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:39 UTC)" (scheduled at 2023-08-08 06:06:29.417660+00:00) gcworker stdout | 2023-08-08 06:06:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265189418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:06:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:06:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:06:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:06:29,881 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:06:30,698 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:06:30,778 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:06:31,213 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:06:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:06:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:54.284093+00:00 (in 22.999659 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:06:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:11:31 UTC)" (scheduled at 2023-08-08 06:06:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:06:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 6, 31, 285017), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:06:31,293 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 6, 31, 285017), True, datetime.datetime(2023, 8, 8, 6, 6, 31, 285017), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:06:31,295 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 6, 31, 285017), True, datetime.datetime(2023, 8, 8, 6, 6, 31, 285017), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 6, 31, 285017), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:06:31,296 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:06:31,296 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:11:31 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:06:31,760 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:06:32,041 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} builder stdout | 2023-08-08 06:06:32,053 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:06:32,064 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:06:32,064 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:06:32,070 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:06:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:42.293757+00:00 (in 9.999348 seconds) chunkcleanupworker stdout | 2023-08-08 06:06:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:42 UTC)" (scheduled at 2023-08-08 06:06:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:06:32,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:06:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 32, 295060), True, datetime.datetime(2023, 8, 8, 6, 6, 32, 295060), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:06:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:06:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:42 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:06:32,389 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-web stdout | 2023-08-08 06:06:32,578 [197] [DEBUG] [app] Starting request: urn:request:b914c4f0-d9fb-4ac8-b679-ac303f067de7 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:06:32,579 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:06:32,579 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:06:32,579 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:06:32,580 [198] [DEBUG] [app] Starting request: urn:request:254e7bd6-103b-4ef5-a83d-0bc00aaead65 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:06:32,580 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:06:32,580 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:06:32,581 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:06:32,586 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:06:32,587 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:06:32,587 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:06:32,587 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:06:32,587 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:06:32,587 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:06:32,587 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:06:32,587 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:06:32,587 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:06:32,588 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:06:32,588 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:06:32,588 [198] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:06:32,588 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:06:32,588 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:06:32,588 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:06:32,588 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:06:32,589 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:06:32,589 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 18, 32, 588644), datetime.datetime(2023, 8, 8, 6, 6, 32, 588689)]) gunicorn-web stdout | 2023-08-08 06:06:32,590 [198] [DEBUG] [app] Ending request: urn:request:254e7bd6-103b-4ef5-a83d-0bc00aaead65 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:254e7bd6-103b-4ef5-a83d-0bc00aaead65', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:06:32,591 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:32,592 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:06:32 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:06:32 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 62 0.013) gunicorn-web stdout | 2023-08-08 06:06:32,593 [197] [DEBUG] [app] Ending request: urn:request:b914c4f0-d9fb-4ac8-b679-ac303f067de7 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:b914c4f0-d9fb-4ac8-b679-ac303f067de7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:06:32,594 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:06:32 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 871 0.018) gunicorn-web stdout | 2023-08-08 06:06:32,594 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:06:32 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-registry stdout | 2023-08-08 06:06:33,214 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:06:33,217 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:06:33,223 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:06:33,226 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:06:33,232 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:06:33,234 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:06:33,238 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:06:33,242 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:06:33,246 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:06:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:06:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:45.371682+00:00 (in 9.999390 seconds) notificationworker stdout | 2023-08-08 06:06:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:45 UTC)" (scheduled at 2023-08-08 06:06:35.371682+00:00) notificationworker stdout | 2023-08-08 06:06:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:06:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 35, 372678), True, datetime.datetime(2023, 8, 8, 6, 6, 35, 372678), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:06:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:06:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:06:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:06:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:06:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:54.393627+00:00 (in 17.813280 seconds) namespacegcworker stdout | 2023-08-08 06:06:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:36 UTC)" (scheduled at 2023-08-08 06:06:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:06:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:06:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:06:37,181 [199] [DEBUG] [app] Starting request: urn:request:69bf2783-7724-400d-8847-fc755f64468b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:06:37,181 [197] [DEBUG] [app] Starting request: urn:request:82b7fca8-131f-4344-9b9e-0e0c2a10064c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:06:37,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:37,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:37,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:37,186 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:06:37,186 [209] [DEBUG] [app] Starting request: urn:request:c13cb901-1dc3-4951-8df5-0b35bae9fb2b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:06:37,187 [209] [DEBUG] [app] Ending request: urn:request:c13cb901-1dc3-4951-8df5-0b35bae9fb2b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c13cb901-1dc3-4951-8df5-0b35bae9fb2b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:06:37,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:06:37,187 [208] [DEBUG] [app] Starting request: urn:request:072bbc94-a84d-4a5d-873a-6b3291e4c046 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:06:37,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:06:37,188 [208] [DEBUG] [app] Ending request: urn:request:072bbc94-a84d-4a5d-873a-6b3291e4c046 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:072bbc94-a84d-4a5d-873a-6b3291e4c046', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:06:37,188 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:06:37,189 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:37,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:37,190 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:37,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:37,193 [197] [DEBUG] [app] Starting request: urn:request:b8605595-f52f-4a8b-aa90-6f9596b518a7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:06:37,193 [197] [DEBUG] [app] Ending request: urn:request:b8605595-f52f-4a8b-aa90-6f9596b518a7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b8605595-f52f-4a8b-aa90-6f9596b518a7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:06:37,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:06:37,193 [198] [DEBUG] [app] Starting request: urn:request:d8168d89-6292-4663-a788-83b4a223c10c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:06:37,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:37,194 [198] [DEBUG] [app] Ending request: urn:request:d8168d89-6292-4663-a788-83b4a223c10c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d8168d89-6292-4663-a788-83b4a223c10c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:06:37,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:06:37,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:06:37,195 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:37,195 [199] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:06:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:06:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:07.197412+00:00 (in 29.999513 seconds) buildlogsarchiver stdout | 2023-08-08 06:06:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:07 UTC)" (scheduled at 2023-08-08 06:06:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:06:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 6, 37, 198374), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:06:37,200 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:06:37,200 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:06:37,206 [197] [DEBUG] [app] Ending request: urn:request:82b7fca8-131f-4344-9b9e-0e0c2a10064c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:82b7fca8-131f-4344-9b9e-0e0c2a10064c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:06:37,206 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:37,206 [199] [DEBUG] [app] Ending request: urn:request:69bf2783-7724-400d-8847-fc755f64468b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:69bf2783-7724-400d-8847-fc755f64468b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:06:37,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:06:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:06:37,206 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:06:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:06:37,207 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:06:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:06:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:06:37,207 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:06:37,209 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:06:37,210 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:06:37,213 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:06:37,213 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:06:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:06:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:49.417660+00:00 (in 9.999431 seconds) gcworker stdout | 2023-08-08 06:06:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:49 UTC)" (scheduled at 2023-08-08 06:06:39.417660+00:00) gcworker stdout | 2023-08-08 06:06:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265199418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:06:39,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:06:39,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:06:39,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:49 UTC)" executed successfully builder stdout | 2023-08-08 06:06:42,080 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:06:42,089 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:06:42,089 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:06:42,094 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:06:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:52.293757+00:00 (in 9.999320 seconds) chunkcleanupworker stdout | 2023-08-08 06:06:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:52 UTC)" (scheduled at 2023-08-08 06:06:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:06:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:06:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 42, 295087), True, datetime.datetime(2023, 8, 8, 6, 6, 42, 295087), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:06:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:06:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:06:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:06:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:55.371682+00:00 (in 9.999467 seconds) notificationworker stdout | 2023-08-08 06:06:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:55 UTC)" (scheduled at 2023-08-08 06:06:45.371682+00:00) notificationworker stdout | 2023-08-08 06:06:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:06:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 45, 372579), True, datetime.datetime(2023, 8, 8, 6, 6, 45, 372579), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:06:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:06:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:06:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:06:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:06:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:25.514341+00:00 (in 38.115287 seconds) exportactionlogsworker stdout | 2023-08-08 06:06:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:47 UTC)" (scheduled at 2023-08-08 06:06:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:06:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:06:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 47, 399577), True, datetime.datetime(2023, 8, 8, 6, 6, 47, 399577), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:06:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:06:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:06:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:06:47,858 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:06:49,082 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:06:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:06:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:59.417660+00:00 (in 9.999422 seconds) gcworker stdout | 2023-08-08 06:06:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:59 UTC)" (scheduled at 2023-08-08 06:06:49.417660+00:00) gcworker stdout | 2023-08-08 06:06:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265209418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:06:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:06:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:06:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:06:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:06:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:06:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:06:56.890741+00:00 (in 7.202841 seconds) securityworker stdout | 2023-08-08 06:06:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:19 UTC)" (scheduled at 2023-08-08 06:06:49.687363+00:00) securityworker stdout | 2023-08-08 06:06:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:06:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:06:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:06:49,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:06:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:06:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:06:50,152 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:06:50,630 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:06:52,101 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:06:52,111 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:06:52,111 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:06:52,115 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:52,179 [200] [DEBUG] [app] Starting request: urn:request:1fd97735-06b1-48f2-b9f4-52b2d0b6bb84 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:06:52,179 [197] [DEBUG] [app] Starting request: urn:request:1e2a1695-f01f-4433-90b6-d7aa0a68fb1b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:06:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:06:52,185 [203] [DEBUG] [app] Starting request: urn:request:382f3c58-8873-4b6f-8939-fb20e81b77c8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:06:52,185 [210] [DEBUG] [app] Starting request: urn:request:463318aa-d853-4a38-b367-80d65db9f4f2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:06:52,186 [203] [DEBUG] [app] Ending request: urn:request:382f3c58-8873-4b6f-8939-fb20e81b77c8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:382f3c58-8873-4b6f-8939-fb20e81b77c8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:06:52,186 [210] [DEBUG] [app] Ending request: urn:request:463318aa-d853-4a38-b367-80d65db9f4f2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:463318aa-d853-4a38-b367-80d65db9f4f2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:06:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:06:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:06:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:06:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:06:52,191 [200] [DEBUG] [app] Starting request: urn:request:e792b498-e5ad-48ea-bd22-04c76ea11f7a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:06:52,192 [199] [DEBUG] [app] Starting request: urn:request:b96f3e73-81a9-442d-801d-bc41b2779679 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:06:52,192 [200] [DEBUG] [app] Ending request: urn:request:e792b498-e5ad-48ea-bd22-04c76ea11f7a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e792b498-e5ad-48ea-bd22-04c76ea11f7a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:06:52,192 [199] [DEBUG] [app] Ending request: urn:request:b96f3e73-81a9-442d-801d-bc41b2779679 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b96f3e73-81a9-442d-801d-bc41b2779679', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:06:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:06:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:06:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:06:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:06:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:06:52,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:06:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:06:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:06:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:06:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:06:52,202 [200] [DEBUG] [app] Ending request: urn:request:1fd97735-06b1-48f2-b9f4-52b2d0b6bb84 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1fd97735-06b1-48f2-b9f4-52b2d0b6bb84', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:06:52,202 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:06:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:06:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:06:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:06:52,203 [197] [DEBUG] [app] Ending request: urn:request:1e2a1695-f01f-4433-90b6-d7aa0a68fb1b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1e2a1695-f01f-4433-90b6-d7aa0a68fb1b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:06:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:06:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:06:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:06:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:06:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:06:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:02.293757+00:00 (in 9.999370 seconds) chunkcleanupworker stdout | 2023-08-08 06:06:52,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:02 UTC)" (scheduled at 2023-08-08 06:06:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:06:52,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:06:52,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 52, 295359), True, datetime.datetime(2023, 8, 8, 6, 6, 52, 295359), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:06:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:06:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:06:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:06:53,348 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:06:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:06:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:13.097375+00:00 (in 18.812583 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:06:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:54 UTC)" (scheduled at 2023-08-08 06:06:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:06:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:06:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:06:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:06:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:36.579841+00:00 (in 42.185681 seconds) namespacegcworker stdout | 2023-08-08 06:06:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:54 UTC)" (scheduled at 2023-08-08 06:06:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:06:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:06:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 54, 394549), True, datetime.datetime(2023, 8, 8, 6, 6, 54, 394549), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:06:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:06:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:06:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:07:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:06:54,962 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:06:54,967 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:06:54,969 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:06:55,188 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} notificationworker stdout | 2023-08-08 06:06:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:06:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:02.178530+00:00 (in 6.806361 seconds) notificationworker stdout | 2023-08-08 06:06:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:05 UTC)" (scheduled at 2023-08-08 06:06:55.371682+00:00) notificationworker stdout | 2023-08-08 06:06:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:06:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 6, 55, 372522), True, datetime.datetime(2023, 8, 8, 6, 6, 55, 372522), 0, 'notification/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:06:55,375 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} notificationworker stdout | 2023-08-08 06:06:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:06:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:06:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:05 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:06:55,573 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:06:56,676 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:06:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:06:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:19.687363+00:00 (in 22.796021 seconds) securityworker stdout | 2023-08-08 06:06:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:26 UTC)" (scheduled at 2023-08-08 06:06:56.890741+00:00) securityworker stdout | 2023-08-08 06:06:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:06:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:06:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:06:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:06:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:06:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:06:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:06:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:06:56,898 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:06:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:06:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 1, 56, 897870), 1, 2]) securityworker stdout | 2023-08-08 06:06:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 1, 56, 897870), 1, 2]) securityworker stdout | 2023-08-08 06:06:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:06:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:06:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:06:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:06:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:06:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:06:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:06:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:06:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:06:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:06:57,214 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:06:57,314 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:06:57,374 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:06:57,440 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:06:57,468 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:06:57,489 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:06:57,493 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} buildlogsarchiver stdout | 2023-08-08 06:06:58,639 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:06:58,815 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:06:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:06:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:09.417660+00:00 (in 9.999416 seconds) gcworker stdout | 2023-08-08 06:06:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:09 UTC)" (scheduled at 2023-08-08 06:06:59.417660+00:00) gcworker stdout | 2023-08-08 06:06:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265219418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:06:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:06:59,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:06:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:06:59,889 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:07:00,735 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:07:00,815 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:07:01,235 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:07:01,778 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:07:02,077 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} builder stdout | 2023-08-08 06:07:02,125 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:07:02,134 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:07:02,134 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:07:02,139 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:07:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:07:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:05.371682+00:00 (in 3.192655 seconds) notificationworker stdout | 2023-08-08 06:07:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:02 UTC)" (scheduled at 2023-08-08 06:07:02.178530+00:00) notificationworker stdout | 2023-08-08 06:07:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:07:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:07:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:07:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:12.293757+00:00 (in 9.999431 seconds) chunkcleanupworker stdout | 2023-08-08 06:07:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:12 UTC)" (scheduled at 2023-08-08 06:07:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:07:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:07:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 2, 294815), True, datetime.datetime(2023, 8, 8, 6, 7, 2, 294815), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:07:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:07:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:12 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:07:02,424 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:07:03,226 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:07:03,229 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:07:03,235 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:07:03,237 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:07:03,241 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:07:03,244 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:07:03,248 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:07:03,251 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:07:03,256 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} expiredappspecifictokenworker stdout | 2023-08-08 06:07:03,572 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2023-08-08 06:07:03,572 [59] [INFO] [apscheduler.executors.default] Running job "ExpiredAppSpecificTokenWorker._gc_expired_tokens (trigger: interval[1:00:00], next run at: 2023-08-08 06:07:03 UTC)" (scheduled at 2023-08-08 06:07:03.572062+00:00) expiredappspecifictokenworker stdout | 2023-08-08 06:07:03,573 [59] [DEBUG] [__main__] Garbage collecting expired app specific tokens with window: 1 day, 0:00:00 expiredappspecifictokenworker stdout | 2023-08-08 06:07:03,573 [59] [DEBUG] [peewee] ('DELETE FROM "appspecificauthtoken" WHERE ("appspecificauthtoken"."expiration" < %s)', [datetime.datetime(2023, 8, 7, 6, 7, 3, 573306)]) expiredappspecifictokenworker stdout | 2023-08-08 06:07:03,574 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 07:07:03.572062+00:00 (in 3599.999674 seconds) expiredappspecifictokenworker stdout | 2023-08-08 06:07:03,580 [59] [DEBUG] [data.database] Disconnecting from database. expiredappspecifictokenworker stdout | 2023-08-08 06:07:03,580 [59] [INFO] [apscheduler.executors.default] Job "ExpiredAppSpecificTokenWorker._gc_expired_tokens (trigger: interval[1:00:00], next run at: 2023-08-08 07:07:03 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:07:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:07:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:09.793658+00:00 (in 5.605656 seconds) repositorygcworker stdout | 2023-08-08 06:07:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:04 UTC)" (scheduled at 2023-08-08 06:07:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:07:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:07:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:07:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:07:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:15.371682+00:00 (in 9.999474 seconds) notificationworker stdout | 2023-08-08 06:07:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:15 UTC)" (scheduled at 2023-08-08 06:07:05.371682+00:00) notificationworker stdout | 2023-08-08 06:07:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:07:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 5, 372802), True, datetime.datetime(2023, 8, 8, 6, 7, 5, 372802), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:07:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:07:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:07:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:07:07,179 [197] [DEBUG] [app] Starting request: urn:request:2f300ce7-78c7-4617-bddb-00bcf2af8cce (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:07:07,179 [200] [DEBUG] [app] Starting request: urn:request:99ba41bf-2d85-41af-8537-546835eee7a5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:07:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:07:07,185 [207] [DEBUG] [app] Starting request: urn:request:d38b2940-5b1c-4a0d-b0c7-958053350002 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:07:07,185 [210] [DEBUG] [app] Starting request: urn:request:ad9efd96-a11c-40fa-add2-9f7889e9052e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:07:07,186 [207] [DEBUG] [app] Ending request: urn:request:d38b2940-5b1c-4a0d-b0c7-958053350002 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d38b2940-5b1c-4a0d-b0c7-958053350002', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:07:07,186 [210] [DEBUG] [app] Ending request: urn:request:ad9efd96-a11c-40fa-add2-9f7889e9052e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ad9efd96-a11c-40fa-add2-9f7889e9052e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:07:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:07:07,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:07:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:07,191 [200] [DEBUG] [app] Starting request: urn:request:a0c1f146-2319-4069-a7e9-87173792ad2e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:07:07,191 [200] [DEBUG] [app] Ending request: urn:request:a0c1f146-2319-4069-a7e9-87173792ad2e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a0c1f146-2319-4069-a7e9-87173792ad2e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:07:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:07,192 [198] [DEBUG] [app] Starting request: urn:request:cb317597-734b-464b-9921-23cee2165e48 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:07:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:07:07,192 [198] [DEBUG] [app] Ending request: urn:request:cb317597-734b-464b-9921-23cee2165e48 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:cb317597-734b-464b-9921-23cee2165e48', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:07:07,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:07:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:07:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:37.197412+00:00 (in 29.999503 seconds) gunicorn-web stdout | 2023-08-08 06:07:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:07:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:37 UTC)" (scheduled at 2023-08-08 06:07:07.197412+00:00) gunicorn-web stdout | 2023-08-08 06:07:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:07:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 7, 7, 198402), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:07:07,203 [200] [DEBUG] [app] Ending request: urn:request:99ba41bf-2d85-41af-8537-546835eee7a5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:99ba41bf-2d85-41af-8537-546835eee7a5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:07:07,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:07:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:07:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:07:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:07:07,204 [197] [DEBUG] [app] Ending request: urn:request:2f300ce7-78c7-4617-bddb-00bcf2af8cce (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2f300ce7-78c7-4617-bddb-00bcf2af8cce', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:07:07,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:07:07,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:07:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:07:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:07:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:07:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:07:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:07:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:07:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:07:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:07:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:19.417660+00:00 (in 9.999454 seconds) gcworker stdout | 2023-08-08 06:07:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:19 UTC)" (scheduled at 2023-08-08 06:07:09.417660+00:00) gcworker stdout | 2023-08-08 06:07:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265229418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:07:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:07:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:07:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:07:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:07:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:04.187429+00:00 (in 54.393197 seconds) repositorygcworker stdout | 2023-08-08 06:07:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:09 UTC)" (scheduled at 2023-08-08 06:07:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:07:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:07:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 9, 794853), True, datetime.datetime(2023, 8, 8, 6, 7, 9, 794853), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:07:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:07:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:07:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:09 UTC)" executed successfully builder stdout | 2023-08-08 06:07:12,139 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:07:12,149 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:07:12,149 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:07:12,153 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:07:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:22.293757+00:00 (in 9.999444 seconds) chunkcleanupworker stdout | 2023-08-08 06:07:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:22 UTC)" (scheduled at 2023-08-08 06:07:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:07:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:07:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 12, 294889), True, datetime.datetime(2023, 8, 8, 6, 7, 12, 294889), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:07:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:07:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:07:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:07:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:54.284093+00:00 (in 41.186141 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:07:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:13 UTC)" (scheduled at 2023-08-08 06:07:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:07:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:07:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 13, 98594), True, datetime.datetime(2023, 8, 8, 6, 7, 13, 98594), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:07:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:07:13,108 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:07:13,108 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:07:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:07:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:25.371682+00:00 (in 9.999438 seconds) notificationworker stdout | 2023-08-08 06:07:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:25 UTC)" (scheduled at 2023-08-08 06:07:15.371682+00:00) notificationworker stdout | 2023-08-08 06:07:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:07:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 15, 372762), True, datetime.datetime(2023, 8, 8, 6, 7, 15, 372762), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:07:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:07:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:07:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:07:17,896 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:07:19,111 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:07:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:07:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:29.417660+00:00 (in 9.999422 seconds) gcworker stdout | 2023-08-08 06:07:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:29 UTC)" (scheduled at 2023-08-08 06:07:19.417660+00:00) gcworker stdout | 2023-08-08 06:07:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265239418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:07:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:07:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:07:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:07:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:07:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:26.890741+00:00 (in 7.202917 seconds) securityworker stdout | 2023-08-08 06:07:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:49 UTC)" (scheduled at 2023-08-08 06:07:19.687363+00:00) securityworker stdout | 2023-08-08 06:07:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:07:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:07:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:07:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:07:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:07:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:07:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 2, 19, 692013), 1, 2]) securityworker stdout | 2023-08-08 06:07:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 2, 19, 692013), 1, 2]) securityworker stdout | 2023-08-08 06:07:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:07:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:07:20,170 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:07:20,644 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} builder stdout | 2023-08-08 06:07:22,163 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:07:22,172 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:07:22,172 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:07:22,177 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:07:22,179 [198] [DEBUG] [app] Starting request: urn:request:bf1c85da-b362-4cdf-ae8e-680c9105b388 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:07:22,179 [197] [DEBUG] [app] Starting request: urn:request:a48bacec-9a32-4a54-8744-227c356abe3a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:07:22,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:22,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:07:22,184 [203] [DEBUG] [app] Starting request: urn:request:d8dc79b5-d4f7-4eaa-a4fb-0f3fd57c9976 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:07:22,185 [208] [DEBUG] [app] Starting request: urn:request:c37b748e-f2ab-4a3b-980f-de4b30ad9d95 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:07:22,185 [203] [DEBUG] [app] Ending request: urn:request:d8dc79b5-d4f7-4eaa-a4fb-0f3fd57c9976 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d8dc79b5-d4f7-4eaa-a4fb-0f3fd57c9976', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:07:22,185 [208] [DEBUG] [app] Ending request: urn:request:c37b748e-f2ab-4a3b-980f-de4b30ad9d95 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c37b748e-f2ab-4a3b-980f-de4b30ad9d95', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:07:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:07:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:07:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:22,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:22,191 [198] [DEBUG] [app] Starting request: urn:request:fd20ff06-8fc3-4268-82de-dce4c1cfc27e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:07:22,191 [198] [DEBUG] [app] Ending request: urn:request:fd20ff06-8fc3-4268-82de-dce4c1cfc27e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fd20ff06-8fc3-4268-82de-dce4c1cfc27e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:07:22,192 [200] [DEBUG] [app] Starting request: urn:request:7c0ef846-2065-4f44-832a-9e2266081ddc (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:07:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:22,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:22,192 [200] [DEBUG] [app] Ending request: urn:request:7c0ef846-2065-4f44-832a-9e2266081ddc (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7c0ef846-2065-4f44-832a-9e2266081ddc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:07:22,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:07:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:07:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:07:22,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:07:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:07:22,203 [197] [DEBUG] [app] Ending request: urn:request:a48bacec-9a32-4a54-8744-227c356abe3a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a48bacec-9a32-4a54-8744-227c356abe3a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:07:22,203 [198] [DEBUG] [app] Ending request: urn:request:bf1c85da-b362-4cdf-ae8e-680c9105b388 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bf1c85da-b362-4cdf-ae8e-680c9105b388', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:07:22,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:07:22,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:07:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:07:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:07:22,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:07:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:07:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:07:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:07:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:07:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:28.874906+00:00 (in 6.580572 seconds) chunkcleanupworker stdout | 2023-08-08 06:07:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:32 UTC)" (scheduled at 2023-08-08 06:07:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:07:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:07:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 22, 294913), True, datetime.datetime(2023, 8, 8, 6, 7, 22, 294913), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:07:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:07:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:07:23,387 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:07:24,971 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:07:24,975 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:07:24,978 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:07:25,223 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} notificationworker stdout | 2023-08-08 06:07:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:07:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:35.371682+00:00 (in 9.999447 seconds) notificationworker stdout | 2023-08-08 06:07:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:35 UTC)" (scheduled at 2023-08-08 06:07:25.371682+00:00) notificationworker stdout | 2023-08-08 06:07:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:07:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 25, 372747), True, datetime.datetime(2023, 8, 8, 6, 7, 25, 372747), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:07:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:07:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:07:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:35 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:07:25,401 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} exportactionlogsworker stdout | 2023-08-08 06:07:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:07:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:47.398551+00:00 (in 21.883689 seconds) exportactionlogsworker stdout | 2023-08-08 06:07:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:25 UTC)" (scheduled at 2023-08-08 06:07:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:07:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:07:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:25 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:07:25,610 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:07:26,712 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:07:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:07:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:49.687363+00:00 (in 22.795790 seconds) securityworker stdout | 2023-08-08 06:07:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:56 UTC)" (scheduled at 2023-08-08 06:07:26.890741+00:00) securityworker stdout | 2023-08-08 06:07:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:07:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:07:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:07:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:07:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:07:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:07:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:07:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:07:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:07:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:07:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 2, 26, 896780), 1, 2]) securityworker stdout | 2023-08-08 06:07:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 2, 26, 896780), 1, 2]) securityworker stdout | 2023-08-08 06:07:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:07:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:07:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:07:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:07:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:07:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:07:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:07:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:07:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:07:27,234 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:07:27,337 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:07:27,383 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:07:27,452 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:07:27,480 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:07:27,504 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:07:27,507 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:07:28,677 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:07:28,849 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} chunkcleanupworker stdout | 2023-08-08 06:07:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:07:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:32.293757+00:00 (in 3.418208 seconds) chunkcleanupworker stdout | 2023-08-08 06:07:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:28 UTC)" (scheduled at 2023-08-08 06:07:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:07:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:07:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:07:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:07:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:39.417660+00:00 (in 9.999425 seconds) gcworker stdout | 2023-08-08 06:07:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:39 UTC)" (scheduled at 2023-08-08 06:07:29.417660+00:00) gcworker stdout | 2023-08-08 06:07:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265249418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:07:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:07:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:07:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:07:29,897 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:07:30,773 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:07:30,835 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:07:31,253 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:07:31,811 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:07:32,103 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} builder stdout | 2023-08-08 06:07:32,178 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:07:32,188 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:07:32,188 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:07:32,192 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:07:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:42.293757+00:00 (in 9.999376 seconds) chunkcleanupworker stdout | 2023-08-08 06:07:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:42 UTC)" (scheduled at 2023-08-08 06:07:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:07:32,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:07:32,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 32, 295052), True, datetime.datetime(2023, 8, 8, 6, 7, 32, 295052), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:07:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:07:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:42 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:07:32,434 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:07:33,236 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:07:33,240 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:07:33,248 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:07:33,252 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:07:33,254 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:07:33,258 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:07:33,260 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:07:33,263 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:07:33,267 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:07:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:07:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:45.371682+00:00 (in 9.999463 seconds) notificationworker stdout | 2023-08-08 06:07:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:45 UTC)" (scheduled at 2023-08-08 06:07:35.371682+00:00) notificationworker stdout | 2023-08-08 06:07:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:07:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 35, 372942), True, datetime.datetime(2023, 8, 8, 6, 7, 35, 372942), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:07:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:07:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:07:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:07:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:07:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:54.393627+00:00 (in 17.813237 seconds) namespacegcworker stdout | 2023-08-08 06:07:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:36 UTC)" (scheduled at 2023-08-08 06:07:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:07:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:07:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:07:37,179 [199] [DEBUG] [app] Starting request: urn:request:9947350a-bd9c-45ce-9a8b-a53eaa254481 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:07:37,179 [197] [DEBUG] [app] Starting request: urn:request:95e2c42b-ec1e-4c5e-bdf6-0636d04af7ba (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:07:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:07:37,184 [208] [DEBUG] [app] Starting request: urn:request:784961f8-2e73-45c7-acfb-8fc42176858d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:07:37,185 [207] [DEBUG] [app] Starting request: urn:request:311e6db1-fbce-4591-abf5-97bcee285437 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:07:37,185 [208] [DEBUG] [app] Ending request: urn:request:784961f8-2e73-45c7-acfb-8fc42176858d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:784961f8-2e73-45c7-acfb-8fc42176858d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:07:37,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:07:37,185 [207] [DEBUG] [app] Ending request: urn:request:311e6db1-fbce-4591-abf5-97bcee285437 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:311e6db1-fbce-4591-abf5-97bcee285437', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:07:37,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:37,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:07:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:37,190 [200] [DEBUG] [app] Starting request: urn:request:c9f15f98-46dc-46c9-b0e7-ff3b581f74bd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:07:37,190 [198] [DEBUG] [app] Starting request: urn:request:1925754c-e5ae-4ca1-81df-1a0bcc9dbf49 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:07:37,191 [200] [DEBUG] [app] Ending request: urn:request:c9f15f98-46dc-46c9-b0e7-ff3b581f74bd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c9f15f98-46dc-46c9-b0e7-ff3b581f74bd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:07:37,191 [198] [DEBUG] [app] Ending request: urn:request:1925754c-e5ae-4ca1-81df-1a0bcc9dbf49 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1925754c-e5ae-4ca1-81df-1a0bcc9dbf49', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:07:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:37,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:07:37,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:07:37,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:07:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:07:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:07:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:07.197412+00:00 (in 29.999519 seconds) buildlogsarchiver stdout | 2023-08-08 06:07:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:07 UTC)" (scheduled at 2023-08-08 06:07:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:07:37,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:07:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 7, 37, 198294), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:07:37,202 [197] [DEBUG] [app] Ending request: urn:request:95e2c42b-ec1e-4c5e-bdf6-0636d04af7ba (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:95e2c42b-ec1e-4c5e-bdf6-0636d04af7ba', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:07:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:07:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:07:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:07:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:07:37,203 [199] [DEBUG] [app] Ending request: urn:request:9947350a-bd9c-45ce-9a8b-a53eaa254481 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9947350a-bd9c-45ce-9a8b-a53eaa254481', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:07:37,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:07:37,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:07:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:07:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:07:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:07:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:07:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:07:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:07:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:07:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:07:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:49.417660+00:00 (in 9.999468 seconds) gcworker stdout | 2023-08-08 06:07:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:49 UTC)" (scheduled at 2023-08-08 06:07:39.417660+00:00) gcworker stdout | 2023-08-08 06:07:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265259418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:07:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:07:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:07:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:49 UTC)" executed successfully builder stdout | 2023-08-08 06:07:42,202 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:07:42,212 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:07:42,212 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:07:42,217 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:07:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:52.293757+00:00 (in 9.999425 seconds) chunkcleanupworker stdout | 2023-08-08 06:07:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:52 UTC)" (scheduled at 2023-08-08 06:07:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:07:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:07:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 42, 294914), True, datetime.datetime(2023, 8, 8, 6, 7, 42, 294914), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:07:42,301 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:07:42,301 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:42,301 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:07:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:07:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:55.371682+00:00 (in 9.999485 seconds) notificationworker stdout | 2023-08-08 06:07:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:55 UTC)" (scheduled at 2023-08-08 06:07:45.371682+00:00) notificationworker stdout | 2023-08-08 06:07:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:07:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 45, 373190), True, datetime.datetime(2023, 8, 8, 6, 7, 45, 373190), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:07:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:07:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:07:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:07:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:07:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:25.514341+00:00 (in 38.115281 seconds) exportactionlogsworker stdout | 2023-08-08 06:07:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:47 UTC)" (scheduled at 2023-08-08 06:07:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:07:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:07:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 47, 399523), True, datetime.datetime(2023, 8, 8, 6, 7, 47, 399523), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:07:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:07:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:07:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:07:47,923 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:07:49,148 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:07:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:07:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:59.417660+00:00 (in 9.999507 seconds) gcworker stdout | 2023-08-08 06:07:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:59 UTC)" (scheduled at 2023-08-08 06:07:49.417660+00:00) gcworker stdout | 2023-08-08 06:07:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265269418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:07:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:07:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:07:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:07:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:07:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:07:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:07:56.890741+00:00 (in 7.202829 seconds) securityworker stdout | 2023-08-08 06:07:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:19 UTC)" (scheduled at 2023-08-08 06:07:49.687363+00:00) securityworker stdout | 2023-08-08 06:07:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:07:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:07:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:07:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:07:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:07:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:07:50,206 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:07:50,656 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:07:52,178 [197] [DEBUG] [app] Starting request: urn:request:2565d8a1-64d6-4cb0-9e00-f52e9779cbe1 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:07:52,179 [199] [DEBUG] [app] Starting request: urn:request:5e27d59f-c987-4e17-9dc6-b73e2dea602a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:07:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:52,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:07:52,184 [205] [DEBUG] [app] Starting request: urn:request:c75aceb7-639f-414b-9a78-9583ed402b93 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:07:52,185 [205] [DEBUG] [app] Ending request: urn:request:c75aceb7-639f-414b-9a78-9583ed402b93 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c75aceb7-639f-414b-9a78-9583ed402b93', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:07:52,185 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:07:52,185 [203] [DEBUG] [app] Starting request: urn:request:152c9c03-d8b1-41d1-b635-b5c72dd8b0c9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:07:52,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:07:52,186 [203] [DEBUG] [app] Ending request: urn:request:152c9c03-d8b1-41d1-b635-b5c72dd8b0c9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:152c9c03-d8b1-41d1-b635-b5c72dd8b0c9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-registry stdout | 2023-08-08 06:07:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:52,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:52,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:07:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:07:52,191 [198] [DEBUG] [app] Starting request: urn:request:d5adbc4a-1b5c-45b2-8081-93260bd953bd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:07:52,191 [200] [DEBUG] [app] Starting request: urn:request:b0951561-187e-4b0c-b4a3-43c729b9b812 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:07:52,192 [198] [DEBUG] [app] Ending request: urn:request:d5adbc4a-1b5c-45b2-8081-93260bd953bd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d5adbc4a-1b5c-45b2-8081-93260bd953bd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:07:52,192 [200] [DEBUG] [app] Ending request: urn:request:b0951561-187e-4b0c-b4a3-43c729b9b812 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b0951561-187e-4b0c-b4a3-43c729b9b812', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:07:52,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:07:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:07:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:07:52,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:07:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:07:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:07:52,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:07:52,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:07:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:07:52,202 [199] [DEBUG] [app] Ending request: urn:request:5e27d59f-c987-4e17-9dc6-b73e2dea602a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5e27d59f-c987-4e17-9dc6-b73e2dea602a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:07:52,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:07:52,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:07:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:07:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:07:52,203 [197] [DEBUG] [app] Ending request: urn:request:2565d8a1-64d6-4cb0-9e00-f52e9779cbe1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2565d8a1-64d6-4cb0-9e00-f52e9779cbe1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:07:52,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:07:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:07:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:07:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" builder stdout | 2023-08-08 06:07:52,227 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:07:52,236 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:07:52,236 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:07:52,240 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:07:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:02.293757+00:00 (in 9.999476 seconds) chunkcleanupworker stdout | 2023-08-08 06:07:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:02 UTC)" (scheduled at 2023-08-08 06:07:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:07:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:07:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 52, 294825), True, datetime.datetime(2023, 8, 8, 6, 7, 52, 294825), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:07:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:07:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:07:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:07:53,425 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:07:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:07:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:13.097375+00:00 (in 18.812745 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:07:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:54 UTC)" (scheduled at 2023-08-08 06:07:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:07:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:07:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:07:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:07:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:36.579841+00:00 (in 42.185680 seconds) namespacegcworker stdout | 2023-08-08 06:07:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:54 UTC)" (scheduled at 2023-08-08 06:07:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:07:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:07:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 54, 394521), True, datetime.datetime(2023, 8, 8, 6, 7, 54, 394521), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:07:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:07:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:07:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:08:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:07:54,980 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:07:54,984 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:07:54,987 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:07:55,242 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} notificationworker stdout | 2023-08-08 06:07:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:07:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:02.178530+00:00 (in 6.806322 seconds) notificationworker stdout | 2023-08-08 06:07:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:05 UTC)" (scheduled at 2023-08-08 06:07:55.371682+00:00) notificationworker stdout | 2023-08-08 06:07:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:07:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 7, 55, 372580), True, datetime.datetime(2023, 8, 8, 6, 7, 55, 372580), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:07:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:07:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:07:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:05 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:07:55,433 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:07:55,618 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:07:56,723 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:07:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:07:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:19.687363+00:00 (in 22.796051 seconds) securityworker stdout | 2023-08-08 06:07:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:26 UTC)" (scheduled at 2023-08-08 06:07:56.890741+00:00) securityworker stdout | 2023-08-08 06:07:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:07:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:07:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:07:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:07:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:07:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:07:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:07:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:07:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:07:56,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:07:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 2, 56, 896523), 1, 2]) securityworker stdout | 2023-08-08 06:07:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 2, 56, 896523), 1, 2]) securityworker stdout | 2023-08-08 06:07:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:07:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:07:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:07:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:07:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:07:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:07:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:07:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:07:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:07:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:07:57,270 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:07:57,354 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:07:57,392 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:07:57,463 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:07:57,493 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:07:57,518 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:07:57,521 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:07:58,688 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:07:58,885 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:07:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:07:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:09.417660+00:00 (in 9.999425 seconds) gcworker stdout | 2023-08-08 06:07:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:09 UTC)" (scheduled at 2023-08-08 06:07:59.417660+00:00) gcworker stdout | 2023-08-08 06:07:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265279418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:07:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:07:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:07:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:07:59,918 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:08:00,809 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:08:00,873 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:08:01,289 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:08:01,852 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:08:02,139 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} notificationworker stdout | 2023-08-08 06:08:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:08:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:05.371682+00:00 (in 3.192663 seconds) notificationworker stdout | 2023-08-08 06:08:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:02 UTC)" (scheduled at 2023-08-08 06:08:02.178530+00:00) notificationworker stdout | 2023-08-08 06:08:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:08:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:02 UTC)" executed successfully builder stdout | 2023-08-08 06:08:02,250 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:08:02,259 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:08:02,259 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:08:02,264 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:08:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:08:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:12.293757+00:00 (in 9.999507 seconds) chunkcleanupworker stdout | 2023-08-08 06:08:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:12 UTC)" (scheduled at 2023-08-08 06:08:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:08:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:08:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 2, 294758), True, datetime.datetime(2023, 8, 8, 6, 8, 2, 294758), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:08:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:08:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:08:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:12 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:08:02,458 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:08:03,247 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:08:03,252 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:08:03,260 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:08:03,264 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:08:03,267 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:08:03,271 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:08:03,277 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:08:03,284 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:08:03,287 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:08:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:08:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:09.793658+00:00 (in 5.605614 seconds) repositorygcworker stdout | 2023-08-08 06:08:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:04 UTC)" (scheduled at 2023-08-08 06:08:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:08:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:08:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:08:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:08:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:15.371682+00:00 (in 9.999487 seconds) notificationworker stdout | 2023-08-08 06:08:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:15 UTC)" (scheduled at 2023-08-08 06:08:05.371682+00:00) notificationworker stdout | 2023-08-08 06:08:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:08:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 5, 373057), True, datetime.datetime(2023, 8, 8, 6, 8, 5, 373057), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:08:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:08:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:08:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:08:07,179 [197] [DEBUG] [app] Starting request: urn:request:8295be8f-c672-4b37-888f-a62037ac1141 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:08:07,179 [198] [DEBUG] [app] Starting request: urn:request:99210d25-e387-4603-bf4f-b4626bfb0dd6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:08:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:07,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:07,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:08:07,185 [209] [DEBUG] [app] Starting request: urn:request:9a4adb05-b2b3-446a-81df-54cc5b995678 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:08:07,185 [209] [DEBUG] [app] Ending request: urn:request:9a4adb05-b2b3-446a-81df-54cc5b995678 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9a4adb05-b2b3-446a-81df-54cc5b995678', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:08:07,185 [207] [DEBUG] [app] Starting request: urn:request:f3e5fa2f-2bfd-4912-a143-5914fbd29d40 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:08:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:08:07,186 [207] [DEBUG] [app] Ending request: urn:request:f3e5fa2f-2bfd-4912-a143-5914fbd29d40 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f3e5fa2f-2bfd-4912-a143-5914fbd29d40', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:08:07,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:08:07,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:07,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:07,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:07,191 [199] [DEBUG] [app] Starting request: urn:request:a86e7031-11b8-4a2b-ab59-27e49d254bbd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:08:07,191 [199] [DEBUG] [app] Ending request: urn:request:a86e7031-11b8-4a2b-ab59-27e49d254bbd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a86e7031-11b8-4a2b-ab59-27e49d254bbd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:07,192 [200] [DEBUG] [app] Starting request: urn:request:ee3983c3-e900-41f3-864c-531b1e240309 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:08:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:08:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:07,192 [200] [DEBUG] [app] Ending request: urn:request:ee3983c3-e900-41f3-864c-531b1e240309 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ee3983c3-e900-41f3-864c-531b1e240309', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:08:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:08:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:08:07,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:07,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:08:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:08:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:08:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:37.197412+00:00 (in 29.999501 seconds) gunicorn-web stdout | 2023-08-08 06:08:07,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:08:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:37 UTC)" (scheduled at 2023-08-08 06:08:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:08:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 8, 7, 198300), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:08:07,203 [197] [DEBUG] [app] Ending request: urn:request:8295be8f-c672-4b37-888f-a62037ac1141 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8295be8f-c672-4b37-888f-a62037ac1141', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:08:07,204 [198] [DEBUG] [app] Ending request: urn:request:99210d25-e387-4603-bf4f-b4626bfb0dd6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:99210d25-e387-4603-bf4f-b4626bfb0dd6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:08:07,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:08:07,204 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:08:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:08:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:08:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:08:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:08:07,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:08:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:08:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:08:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:08:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:08:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:08:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:08:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:08:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:19.417660+00:00 (in 9.999489 seconds) gcworker stdout | 2023-08-08 06:08:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:19 UTC)" (scheduled at 2023-08-08 06:08:09.417660+00:00) gcworker stdout | 2023-08-08 06:08:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265289418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:08:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:08:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:08:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:08:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:08:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:04.187429+00:00 (in 54.393078 seconds) repositorygcworker stdout | 2023-08-08 06:08:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:09 UTC)" (scheduled at 2023-08-08 06:08:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:08:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:08:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 9, 794954), True, datetime.datetime(2023, 8, 8, 6, 8, 9, 794954), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:08:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:08:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:08:09,804 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:09 UTC)" executed successfully builder stdout | 2023-08-08 06:08:12,264 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:08:12,273 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:08:12,273 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:08:12,278 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:08:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:08:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:22.293757+00:00 (in 9.999526 seconds) chunkcleanupworker stdout | 2023-08-08 06:08:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:22 UTC)" (scheduled at 2023-08-08 06:08:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:08:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:08:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 12, 294698), True, datetime.datetime(2023, 8, 8, 6, 8, 12, 294698), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:08:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:08:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:08:12,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:08:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:08:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:54.284093+00:00 (in 41.186160 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:08:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:13 UTC)" (scheduled at 2023-08-08 06:08:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:08:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:08:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 13, 98511), True, datetime.datetime(2023, 8, 8, 6, 8, 13, 98511), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:08:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:08:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:08:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:08:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:08:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:25.371682+00:00 (in 9.998941 seconds) notificationworker stdout | 2023-08-08 06:08:15,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:25 UTC)" (scheduled at 2023-08-08 06:08:15.371682+00:00) notificationworker stdout | 2023-08-08 06:08:15,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:08:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 15, 373118), True, datetime.datetime(2023, 8, 8, 6, 8, 15, 373118), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:08:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:08:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:08:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:08:17,940 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:08:19,183 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:08:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:08:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:29.417660+00:00 (in 9.999438 seconds) gcworker stdout | 2023-08-08 06:08:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:29 UTC)" (scheduled at 2023-08-08 06:08:19.417660+00:00) gcworker stdout | 2023-08-08 06:08:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265299418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:08:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:08:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:08:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:08:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:08:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:26.890741+00:00 (in 7.202863 seconds) securityworker stdout | 2023-08-08 06:08:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:49 UTC)" (scheduled at 2023-08-08 06:08:19.687363+00:00) securityworker stdout | 2023-08-08 06:08:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:08:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:08:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:08:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:08:19,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:08:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:08:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 3, 19, 691754), 1, 2]) securityworker stdout | 2023-08-08 06:08:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 3, 19, 691754), 1, 2]) securityworker stdout | 2023-08-08 06:08:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:08:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:08:20,243 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:08:20,692 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:08:22,179 [197] [DEBUG] [app] Starting request: urn:request:1c21707b-7a17-485d-83e2-d9ad04994f6f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:08:22,179 [199] [DEBUG] [app] Starting request: urn:request:9673a80e-a030-409e-920e-7a5a6d5a6ad5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:08:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:22,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:08:22,185 [208] [DEBUG] [app] Starting request: urn:request:58db6cf8-8277-4857-b18c-cf491a1f06fc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:08:22,186 [207] [DEBUG] [app] Starting request: urn:request:efd99363-64f7-43e3-9a02-7a329edc5beb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:08:22,186 [208] [DEBUG] [app] Ending request: urn:request:58db6cf8-8277-4857-b18c-cf491a1f06fc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:58db6cf8-8277-4857-b18c-cf491a1f06fc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:08:22,186 [207] [DEBUG] [app] Ending request: urn:request:efd99363-64f7-43e3-9a02-7a329edc5beb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:efd99363-64f7-43e3-9a02-7a329edc5beb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:08:22,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:08:22,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:08:22,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:22,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:22,192 [198] [DEBUG] [app] Starting request: urn:request:11b85898-a54f-48e2-b36c-8ba71791d959 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:08:22,192 [197] [DEBUG] [app] Starting request: urn:request:62985919-30e2-4f54-97d6-7d9ffcad3e7e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:08:22,192 [198] [DEBUG] [app] Ending request: urn:request:11b85898-a54f-48e2-b36c-8ba71791d959 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:11b85898-a54f-48e2-b36c-8ba71791d959', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:22,192 [197] [DEBUG] [app] Ending request: urn:request:62985919-30e2-4f54-97d6-7d9ffcad3e7e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:62985919-30e2-4f54-97d6-7d9ffcad3e7e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:22,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:08:22,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:08:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:22,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:08:22,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:08:22,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:08:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:08:22,203 [199] [DEBUG] [app] Ending request: urn:request:9673a80e-a030-409e-920e-7a5a6d5a6ad5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9673a80e-a030-409e-920e-7a5a6d5a6ad5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:08:22,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:08:22,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:08:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:08:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:08:22,205 [197] [DEBUG] [app] Ending request: urn:request:1c21707b-7a17-485d-83e2-d9ad04994f6f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1c21707b-7a17-485d-83e2-d9ad04994f6f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:08:22,205 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:08:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:08:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:08:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" builder stdout | 2023-08-08 06:08:22,288 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue chunkcleanupworker stdout | 2023-08-08 06:08:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:08:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:28.874906+00:00 (in 6.580685 seconds) chunkcleanupworker stdout | 2023-08-08 06:08:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:32 UTC)" (scheduled at 2023-08-08 06:08:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:08:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:08:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 22, 294696), True, datetime.datetime(2023, 8, 8, 6, 8, 22, 294696), 0, 'chunk_cleanup/%', 50, 1, 0]) builder stdout | 2023-08-08 06:08:22,297 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:08:22,297 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:08:22,301 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:08:22,301 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:08:22,301 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:08:22,301 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:08:23,462 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:08:24,989 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:08:24,992 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:08:24,995 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:08:25,257 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} notificationworker stdout | 2023-08-08 06:08:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:08:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:35.371682+00:00 (in 9.999459 seconds) notificationworker stdout | 2023-08-08 06:08:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:35 UTC)" (scheduled at 2023-08-08 06:08:25.371682+00:00) notificationworker stdout | 2023-08-08 06:08:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:08:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 25, 372604), True, datetime.datetime(2023, 8, 8, 6, 8, 25, 372604), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:08:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:08:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:08:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:35 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:08:25,454 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} exportactionlogsworker stdout | 2023-08-08 06:08:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:08:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:33.514094+00:00 (in 7.999205 seconds) exportactionlogsworker stdout | 2023-08-08 06:08:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:25 UTC)" (scheduled at 2023-08-08 06:08:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:08:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:08:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:25 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:08:25,655 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:08:26,760 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:08:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:08:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:49.687363+00:00 (in 22.796117 seconds) securityworker stdout | 2023-08-08 06:08:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:56 UTC)" (scheduled at 2023-08-08 06:08:26.890741+00:00) securityworker stdout | 2023-08-08 06:08:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:08:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:08:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:08:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:08:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:08:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:08:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:08:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:08:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:08:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:08:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 3, 26, 896436), 1, 2]) securityworker stdout | 2023-08-08 06:08:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 3, 26, 896436), 1, 2]) securityworker stdout | 2023-08-08 06:08:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:08:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:08:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:08:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:08:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:08:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:08:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:08:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:08:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:08:27,307 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:08:27,388 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:08:27,400 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:08:27,475 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:08:27,505 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:08:27,531 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:08:27,533 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:08:28,725 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} chunkcleanupworker stdout | 2023-08-08 06:08:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:08:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:32.293757+00:00 (in 3.418346 seconds) chunkcleanupworker stdout | 2023-08-08 06:08:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:28 UTC)" (scheduled at 2023-08-08 06:08:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:08:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:08:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:28 UTC)" executed successfully builder stdout | 2023-08-08 06:08:28,894 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:08:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:08:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:39.417660+00:00 (in 9.999405 seconds) gcworker stdout | 2023-08-08 06:08:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:39 UTC)" (scheduled at 2023-08-08 06:08:29.417660+00:00) gcworker stdout | 2023-08-08 06:08:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265309418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:08:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:08:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:08:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:08:29,948 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:08:30,849 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:08:30,903 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:08:31,325 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:08:31,872 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:08:32,177 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} chunkcleanupworker stdout | 2023-08-08 06:08:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:08:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:42.293757+00:00 (in 9.999445 seconds) chunkcleanupworker stdout | 2023-08-08 06:08:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:42 UTC)" (scheduled at 2023-08-08 06:08:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:08:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:08:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 32, 294898), True, datetime.datetime(2023, 8, 8, 6, 8, 32, 294898), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:08:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:08:32,303 [57] [DEBUG] [data.database] Disconnecting from database. builder stdout | 2023-08-08 06:08:32,303 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue chunkcleanupworker stdout | 2023-08-08 06:08:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:42 UTC)" executed successfully builder stdout | 2023-08-08 06:08:32,312 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:08:32,312 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:08:32,317 [55] [DEBUG] [data.database] Disconnecting from database. expiredappspecifictokenworker stdout | 2023-08-08 06:08:32,497 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:08:33,260 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:08:33,264 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:08:33,268 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:08:33,276 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:08:33,279 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:08:33,282 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:08:33,285 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:08:33,293 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:08:33,296 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} exportactionlogsworker stdout | 2023-08-08 06:08:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:08:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:47.398551+00:00 (in 13.883884 seconds) exportactionlogsworker stdout | 2023-08-08 06:08:33,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:13:33 UTC)" (scheduled at 2023-08-08 06:08:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:08:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 8, 33, 515192), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:08:33,523 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 8, 33, 515192), True, datetime.datetime(2023, 8, 8, 6, 8, 33, 515192), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:08:33,524 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 8, 33, 515192), True, datetime.datetime(2023, 8, 8, 6, 8, 33, 515192), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 8, 33, 515192), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:08:33,526 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:08:33,526 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:13:33 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:08:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:08:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:45.371682+00:00 (in 9.999429 seconds) notificationworker stdout | 2023-08-08 06:08:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:45 UTC)" (scheduled at 2023-08-08 06:08:35.371682+00:00) notificationworker stdout | 2023-08-08 06:08:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:08:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 35, 372633), True, datetime.datetime(2023, 8, 8, 6, 8, 35, 372633), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:08:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:08:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:08:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:08:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:08:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:54.393627+00:00 (in 17.813228 seconds) namespacegcworker stdout | 2023-08-08 06:08:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:36 UTC)" (scheduled at 2023-08-08 06:08:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:08:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:08:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:08:37,179 [200] [DEBUG] [app] Starting request: urn:request:b88b3cbc-d6cc-42d5-9218-5f1ea998579f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:08:37,179 [197] [DEBUG] [app] Starting request: urn:request:8346041a-9eb7-47b6-9262-6de7edd0d317 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:08:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:08:37,184 [203] [DEBUG] [app] Starting request: urn:request:49606744-dc58-46b9-8d84-5ed1acfe11de (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:08:37,185 [203] [DEBUG] [app] Ending request: urn:request:49606744-dc58-46b9-8d84-5ed1acfe11de (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:49606744-dc58-46b9-8d84-5ed1acfe11de', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:08:37,185 [207] [DEBUG] [app] Starting request: urn:request:052738ca-f5fa-4c7f-8388-d49b50fbd1e1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:08:37,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:08:37,186 [207] [DEBUG] [app] Ending request: urn:request:052738ca-f5fa-4c7f-8388-d49b50fbd1e1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:052738ca-f5fa-4c7f-8388-d49b50fbd1e1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:08:37,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:08:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:37,190 [198] [DEBUG] [app] Starting request: urn:request:04347817-1443-4303-ae94-51c7e944884e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:08:37,191 [198] [DEBUG] [app] Ending request: urn:request:04347817-1443-4303-ae94-51c7e944884e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:04347817-1443-4303-ae94-51c7e944884e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:37,191 [200] [DEBUG] [app] Starting request: urn:request:447e45f3-7a0c-4457-a4e1-03e716d0a796 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:08:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:08:37,192 [200] [DEBUG] [app] Ending request: urn:request:447e45f3-7a0c-4457-a4e1-03e716d0a796 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:447e45f3-7a0c-4457-a4e1-03e716d0a796', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:08:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:08:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:08:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:08:37,198 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:08:37,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:08:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:07.197412+00:00 (in 29.999054 seconds) buildlogsarchiver stdout | 2023-08-08 06:08:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:07 UTC)" (scheduled at 2023-08-08 06:08:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:08:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 8, 37, 198715), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:08:37,203 [197] [DEBUG] [app] Ending request: urn:request:8346041a-9eb7-47b6-9262-6de7edd0d317 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8346041a-9eb7-47b6-9262-6de7edd0d317', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:08:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:08:37,203 [200] [DEBUG] [app] Ending request: urn:request:b88b3cbc-d6cc-42d5-9218-5f1ea998579f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b88b3cbc-d6cc-42d5-9218-5f1ea998579f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:08:37,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:08:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:08:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:08:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:08:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:08:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:08:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:08:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:08:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:08:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:08:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:08:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:08:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:08:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:49.417660+00:00 (in 9.999378 seconds) gcworker stdout | 2023-08-08 06:08:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:49 UTC)" (scheduled at 2023-08-08 06:08:39.417660+00:00) gcworker stdout | 2023-08-08 06:08:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265319418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:08:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:08:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:08:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:08:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:08:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:52.293757+00:00 (in 9.999445 seconds) chunkcleanupworker stdout | 2023-08-08 06:08:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:52 UTC)" (scheduled at 2023-08-08 06:08:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:08:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:08:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 42, 294825), True, datetime.datetime(2023, 8, 8, 6, 8, 42, 294825), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:08:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:08:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:08:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:52 UTC)" executed successfully builder stdout | 2023-08-08 06:08:42,327 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:08:42,335 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:08:42,335 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:08:42,340 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:08:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:08:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:49.178248+00:00 (in 3.806010 seconds) notificationworker stdout | 2023-08-08 06:08:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:55 UTC)" (scheduled at 2023-08-08 06:08:45.371682+00:00) notificationworker stdout | 2023-08-08 06:08:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:08:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 45, 372594), True, datetime.datetime(2023, 8, 8, 6, 8, 45, 372594), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:08:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:08:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:08:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:08:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:08:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:25.514341+00:00 (in 38.115238 seconds) exportactionlogsworker stdout | 2023-08-08 06:08:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:47 UTC)" (scheduled at 2023-08-08 06:08:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:08:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:08:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 47, 399563), True, datetime.datetime(2023, 8, 8, 6, 8, 47, 399563), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:08:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:08:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:08:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:08:47,978 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} notificationworker stdout | 2023-08-08 06:08:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:08:49,178 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:55.371682+00:00 (in 6.192899 seconds) notificationworker stdout | 2023-08-08 06:08:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:13:49 UTC)" (scheduled at 2023-08-08 06:08:49.178248+00:00) notificationworker stdout | 2023-08-08 06:08:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 8, 49, 179105), 'notification/%']) notificationworker stdout | 2023-08-08 06:08:49,186 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 8, 49, 179105), True, datetime.datetime(2023, 8, 8, 6, 8, 49, 179105), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:08:49,188 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 8, 49, 179105), True, datetime.datetime(2023, 8, 8, 6, 8, 49, 179105), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 8, 49, 179105), 'notification/%']) notificationworker stdout | 2023-08-08 06:08:49,189 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:08:49,189 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:13:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:08:49,219 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:08:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:08:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:59.417660+00:00 (in 9.999421 seconds) gcworker stdout | 2023-08-08 06:08:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:59 UTC)" (scheduled at 2023-08-08 06:08:49.417660+00:00) gcworker stdout | 2023-08-08 06:08:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265329418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:08:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:08:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:08:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:08:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:08:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:08:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:08:56.890741+00:00 (in 7.202887 seconds) securityworker stdout | 2023-08-08 06:08:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:19 UTC)" (scheduled at 2023-08-08 06:08:49.687363+00:00) securityworker stdout | 2023-08-08 06:08:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:08:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:08:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:08:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:08:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:08:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:08:50,281 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:08:50,722 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:08:52,179 [198] [DEBUG] [app] Starting request: urn:request:db5d9bd1-d97f-4e17-8c4f-428cc6fc20d3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:08:52,180 [200] [DEBUG] [app] Starting request: urn:request:68e8238a-e096-4966-8564-dbd35d8e0416 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:08:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:08:52,185 [209] [DEBUG] [app] Starting request: urn:request:5a4c4935-a504-49e4-bbac-42f1e6e22bc2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:08:52,185 [203] [DEBUG] [app] Starting request: urn:request:96d7806f-9aa3-478d-895f-baaaaa3f4c05 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:08:52,186 [209] [DEBUG] [app] Ending request: urn:request:5a4c4935-a504-49e4-bbac-42f1e6e22bc2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5a4c4935-a504-49e4-bbac-42f1e6e22bc2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:08:52,186 [203] [DEBUG] [app] Ending request: urn:request:96d7806f-9aa3-478d-895f-baaaaa3f4c05 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:96d7806f-9aa3-478d-895f-baaaaa3f4c05', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:08:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:08:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:08:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:52,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:08:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:52,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:08:52,191 [197] [DEBUG] [app] Starting request: urn:request:a1629152-9a6e-4108-a52c-ddabd95d2cac (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:08:52,191 [197] [DEBUG] [app] Ending request: urn:request:a1629152-9a6e-4108-a52c-ddabd95d2cac (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a1629152-9a6e-4108-a52c-ddabd95d2cac', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:08:52,192 [199] [DEBUG] [app] Starting request: urn:request:987858b5-358a-4f1d-a124-ec80d5c618ef (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:08:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:52,192 [199] [DEBUG] [app] Ending request: urn:request:987858b5-358a-4f1d-a124-ec80d5c618ef (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:987858b5-358a-4f1d-a124-ec80d5c618ef', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:08:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:08:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:08:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:08:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:08:52,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:08:52,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:08:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:08:52,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:08:52,202 [200] [DEBUG] [app] Ending request: urn:request:68e8238a-e096-4966-8564-dbd35d8e0416 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:68e8238a-e096-4966-8564-dbd35d8e0416', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:08:52,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:08:52,203 [198] [DEBUG] [app] Ending request: urn:request:db5d9bd1-d97f-4e17-8c4f-428cc6fc20d3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:db5d9bd1-d97f-4e17-8c4f-428cc6fc20d3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:08:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:08:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:08:52,203 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:08:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:08:52,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:08:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:08:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) chunkcleanupworker stdout | 2023-08-08 06:08:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:08:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:02.293757+00:00 (in 9.999453 seconds) chunkcleanupworker stdout | 2023-08-08 06:08:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:02 UTC)" (scheduled at 2023-08-08 06:08:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:08:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:08:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 52, 294873), True, datetime.datetime(2023, 8, 8, 6, 8, 52, 294873), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:08:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:08:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:08:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:02 UTC)" executed successfully builder stdout | 2023-08-08 06:08:52,350 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:08:52,360 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:08:52,360 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:08:52,366 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:08:53,500 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:08:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:08:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:13.097375+00:00 (in 18.812640 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:08:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:54 UTC)" (scheduled at 2023-08-08 06:08:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:08:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:08:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:08:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:08:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:36.579841+00:00 (in 42.185638 seconds) namespacegcworker stdout | 2023-08-08 06:08:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:54 UTC)" (scheduled at 2023-08-08 06:08:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:08:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:08:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 54, 394704), True, datetime.datetime(2023, 8, 8, 6, 8, 54, 394704), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:08:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:08:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:08:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:09:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:08:54,998 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:08:55,001 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:08:55,004 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:08:55,283 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} notificationworker stdout | 2023-08-08 06:08:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:08:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:02.178530+00:00 (in 6.806331 seconds) notificationworker stdout | 2023-08-08 06:08:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:05 UTC)" (scheduled at 2023-08-08 06:08:55.371682+00:00) notificationworker stdout | 2023-08-08 06:08:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:08:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 8, 55, 372553), True, datetime.datetime(2023, 8, 8, 6, 8, 55, 372553), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:08:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:08:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:08:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:05 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:08:55,491 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:08:55,694 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:08:56,798 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:08:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:08:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:19.687363+00:00 (in 22.796079 seconds) securityworker stdout | 2023-08-08 06:08:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:26 UTC)" (scheduled at 2023-08-08 06:08:56.890741+00:00) securityworker stdout | 2023-08-08 06:08:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:08:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:08:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:08:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:08:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:08:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:08:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:08:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:08:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:08:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:08:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 3, 56, 896559), 1, 2]) securityworker stdout | 2023-08-08 06:08:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 3, 56, 896559), 1, 2]) securityworker stdout | 2023-08-08 06:08:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:08:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:08:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:08:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:08:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:08:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:08:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:08:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:08:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:08:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:08:57,346 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:08:57,401 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:08:57,409 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:08:57,486 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:08:57,518 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:08:57,543 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:08:57,547 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:08:58,732 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:08:58,922 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:08:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:08:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:09.417660+00:00 (in 9.999425 seconds) gcworker stdout | 2023-08-08 06:08:59,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:09 UTC)" (scheduled at 2023-08-08 06:08:59.417660+00:00) gcworker stdout | 2023-08-08 06:08:59,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265339419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:08:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:08:59,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:08:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:08:59,985 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:09:00,878 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:09:00,933 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:09:01,363 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:09:01,898 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} notificationworker stdout | 2023-08-08 06:09:02,179 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:09:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:05.371682+00:00 (in 3.192493 seconds) notificationworker stdout | 2023-08-08 06:09:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:02 UTC)" (scheduled at 2023-08-08 06:09:02.178530+00:00) notificationworker stdout | 2023-08-08 06:09:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:09:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:02 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:09:02,215 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} chunkcleanupworker stdout | 2023-08-08 06:09:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:09:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:12.293757+00:00 (in 9.999462 seconds) chunkcleanupworker stdout | 2023-08-08 06:09:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:12 UTC)" (scheduled at 2023-08-08 06:09:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:09:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:09:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 2, 294810), True, datetime.datetime(2023, 8, 8, 6, 9, 2, 294810), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:09:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:09:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:09:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:12 UTC)" executed successfully builder stdout | 2023-08-08 06:09:02,371 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:09:02,380 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:09:02,380 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:09:02,384 [55] [DEBUG] [data.database] Disconnecting from database. expiredappspecifictokenworker stdout | 2023-08-08 06:09:02,522 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:09:03,270 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:09:03,274 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:09:03,277 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:09:03,287 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:09:03,290 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:09:03,293 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:09:03,296 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:09:03,304 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:09:03,307 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:09:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:09:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:09.793658+00:00 (in 5.605651 seconds) repositorygcworker stdout | 2023-08-08 06:09:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:04 UTC)" (scheduled at 2023-08-08 06:09:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:09:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:09:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:09:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:09:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:15.371682+00:00 (in 9.999461 seconds) notificationworker stdout | 2023-08-08 06:09:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:15 UTC)" (scheduled at 2023-08-08 06:09:05.371682+00:00) notificationworker stdout | 2023-08-08 06:09:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:09:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 5, 372739), True, datetime.datetime(2023, 8, 8, 6, 9, 5, 372739), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:09:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:09:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:09:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:09:07,179 [199] [DEBUG] [app] Starting request: urn:request:a5161323-3126-4a01-9ee3-8a9fcfd3a607 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:09:07,180 [200] [DEBUG] [app] Starting request: urn:request:ba0e63ff-c19d-484e-bbba-306f710c58cc (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:09:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:07,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:07,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:09:07,185 [209] [DEBUG] [app] Starting request: urn:request:7d33796b-ef60-4357-b249-95f98733e234 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:09:07,186 [207] [DEBUG] [app] Starting request: urn:request:94d64686-c652-4a14-a430-e9c7901af9f7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:09:07,186 [209] [DEBUG] [app] Ending request: urn:request:7d33796b-ef60-4357-b249-95f98733e234 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7d33796b-ef60-4357-b249-95f98733e234', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:09:07,186 [207] [DEBUG] [app] Ending request: urn:request:94d64686-c652-4a14-a430-e9c7901af9f7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:94d64686-c652-4a14-a430-e9c7901af9f7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:09:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:09:07,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:07,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:09:07,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:09:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:07,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:07,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:07,192 [197] [DEBUG] [app] Starting request: urn:request:99f4bf19-52c0-491a-9621-77ae8ef0b150 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:09:07,192 [200] [DEBUG] [app] Starting request: urn:request:d1a722bf-39de-4deb-a77e-99fc1ea2ebd1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:09:07,193 [197] [DEBUG] [app] Ending request: urn:request:99f4bf19-52c0-491a-9621-77ae8ef0b150 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:99f4bf19-52c0-491a-9621-77ae8ef0b150', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:09:07,193 [200] [DEBUG] [app] Ending request: urn:request:d1a722bf-39de-4deb-a77e-99fc1ea2ebd1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d1a722bf-39de-4deb-a77e-99fc1ea2ebd1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:09:07,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:07,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:09:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:09:07,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:09:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:09:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:09:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:37.197412+00:00 (in 29.999519 seconds) buildlogsarchiver stdout | 2023-08-08 06:09:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:37 UTC)" (scheduled at 2023-08-08 06:09:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:09:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 9, 7, 198364), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:09:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:09:07,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:09:07,204 [199] [DEBUG] [app] Ending request: urn:request:a5161323-3126-4a01-9ee3-8a9fcfd3a607 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a5161323-3126-4a01-9ee3-8a9fcfd3a607', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:09:07,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:09:07,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:09:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:09:07,205 [200] [DEBUG] [app] Ending request: urn:request:ba0e63ff-c19d-484e-bbba-306f710c58cc (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ba0e63ff-c19d-484e-bbba-306f710c58cc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:09:07,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:09:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 06:09:07,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:09:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:09:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:09:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:09:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:09:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:09:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:09:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:09:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:09:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:19.417660+00:00 (in 9.999436 seconds) gcworker stdout | 2023-08-08 06:09:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:19 UTC)" (scheduled at 2023-08-08 06:09:09.417660+00:00) gcworker stdout | 2023-08-08 06:09:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265349418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:09:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:09:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:09:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:09:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:09:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:19.187230+00:00 (in 9.392975 seconds) repositorygcworker stdout | 2023-08-08 06:09:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:09 UTC)" (scheduled at 2023-08-08 06:09:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:09:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:09:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 9, 795318), True, datetime.datetime(2023, 8, 8, 6, 9, 9, 795318), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:09:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:09:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:09:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:09:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:09:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:22.293757+00:00 (in 9.999477 seconds) chunkcleanupworker stdout | 2023-08-08 06:09:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:22 UTC)" (scheduled at 2023-08-08 06:09:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:09:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:09:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 12, 294849), True, datetime.datetime(2023, 8, 8, 6, 9, 12, 294849), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:09:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:09:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:09:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:22 UTC)" executed successfully builder stdout | 2023-08-08 06:09:12,394 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:09:12,403 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:09:12,403 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:09:12,408 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:09:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:09:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:54.284093+00:00 (in 41.186109 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:09:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:13 UTC)" (scheduled at 2023-08-08 06:09:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:09:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:09:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 13, 98519), True, datetime.datetime(2023, 8, 8, 6, 9, 13, 98519), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:09:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:09:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:09:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:09:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:09:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:25.371682+00:00 (in 9.999431 seconds) notificationworker stdout | 2023-08-08 06:09:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:25 UTC)" (scheduled at 2023-08-08 06:09:15.371682+00:00) notificationworker stdout | 2023-08-08 06:09:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:09:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 15, 372651), True, datetime.datetime(2023, 8, 8, 6, 9, 15, 372651), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:09:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:09:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:09:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:09:17,985 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} repositorygcworker stdout | 2023-08-08 06:09:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:09:19,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:04.187429+00:00 (in 44.999589 seconds) repositorygcworker stdout | 2023-08-08 06:09:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:14:19 UTC)" (scheduled at 2023-08-08 06:09:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:09:19,189 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 9, 19, 188795), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:09:19,196 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 9, 19, 188795), True, datetime.datetime(2023, 8, 8, 6, 9, 19, 188795), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:09:19,198 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 9, 19, 188795), True, datetime.datetime(2023, 8, 8, 6, 9, 19, 188795), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 9, 19, 188795), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:09:19,199 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:09:19,200 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:14:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:09:19,252 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:09:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:09:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:29.417660+00:00 (in 9.999450 seconds) gcworker stdout | 2023-08-08 06:09:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:29 UTC)" (scheduled at 2023-08-08 06:09:19.417660+00:00) gcworker stdout | 2023-08-08 06:09:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265359418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:09:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:09:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:09:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:09:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:09:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:26.890741+00:00 (in 7.202832 seconds) securityworker stdout | 2023-08-08 06:09:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:49 UTC)" (scheduled at 2023-08-08 06:09:19.687363+00:00) securityworker stdout | 2023-08-08 06:09:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:09:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:09:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:09:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:09:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:09:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:09:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 4, 19, 691887), 1, 2]) securityworker stdout | 2023-08-08 06:09:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 4, 19, 691887), 1, 2]) securityworker stdout | 2023-08-08 06:09:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:09:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:09:20,318 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:09:20,752 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:09:22,179 [197] [DEBUG] [app] Starting request: urn:request:ec006874-21d8-4f94-a4ec-22c389144746 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:09:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:22,181 [199] [DEBUG] [app] Starting request: urn:request:0ee6396d-250a-47e7-beb5-e2d1c1faec56 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:09:22,183 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:22,186 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:09:22,186 [210] [DEBUG] [app] Starting request: urn:request:d001feef-79ad-44ff-afee-87ca831c3ae4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:09:22,187 [208] [DEBUG] [app] Starting request: urn:request:44379133-590c-4914-96eb-609a8a5e4dba (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:09:22,187 [210] [DEBUG] [app] Ending request: urn:request:d001feef-79ad-44ff-afee-87ca831c3ae4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d001feef-79ad-44ff-afee-87ca831c3ae4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:09:22,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-registry stdout | 2023-08-08 06:09:22,187 [208] [DEBUG] [app] Ending request: urn:request:44379133-590c-4914-96eb-609a8a5e4dba (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:44379133-590c-4914-96eb-609a8a5e4dba', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:09:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:09:22,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:09:22,188 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:09:22,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:22,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:22,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:22,192 [198] [DEBUG] [app] Starting request: urn:request:0885f770-17be-4045-a47c-35adbfb4df72 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:09:22,192 [199] [DEBUG] [app] Starting request: urn:request:0a85e21d-3878-4a94-8cbe-6e64519345ce (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:09:22,193 [198] [DEBUG] [app] Ending request: urn:request:0885f770-17be-4045-a47c-35adbfb4df72 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0885f770-17be-4045-a47c-35adbfb4df72', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:09:22,193 [199] [DEBUG] [app] Ending request: urn:request:0a85e21d-3878-4a94-8cbe-6e64519345ce (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0a85e21d-3878-4a94-8cbe-6e64519345ce', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:09:22,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:22,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:09:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:09:22,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:09:22,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:09:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:09:22,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:09:22,204 [197] [DEBUG] [app] Ending request: urn:request:ec006874-21d8-4f94-a4ec-22c389144746 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ec006874-21d8-4f94-a4ec-22c389144746', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:09:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:09:22,204 [199] [DEBUG] [app] Ending request: urn:request:0ee6396d-250a-47e7-beb5-e2d1c1faec56 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0ee6396d-250a-47e7-beb5-e2d1c1faec56', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:09:22,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:09:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:09:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:09:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:09:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:09:22,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:09:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:09:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:09:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:28.874906+00:00 (in 6.580646 seconds) chunkcleanupworker stdout | 2023-08-08 06:09:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:32 UTC)" (scheduled at 2023-08-08 06:09:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:09:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:09:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 22, 294775), True, datetime.datetime(2023, 8, 8, 6, 9, 22, 294775), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:09:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:09:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:09:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:32 UTC)" executed successfully builder stdout | 2023-08-08 06:09:22,418 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:09:22,427 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:09:22,427 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:09:22,431 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:09:23,522 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:09:25,008 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:09:25,011 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:09:25,014 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:09:25,320 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} notificationworker stdout | 2023-08-08 06:09:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:09:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:35.371682+00:00 (in 9.999481 seconds) notificationworker stdout | 2023-08-08 06:09:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:35 UTC)" (scheduled at 2023-08-08 06:09:25.371682+00:00) notificationworker stdout | 2023-08-08 06:09:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:09:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 25, 372584), True, datetime.datetime(2023, 8, 8, 6, 9, 25, 372584), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:09:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:09:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:09:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:09:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:09:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:47.398551+00:00 (in 21.883648 seconds) exportactionlogsworker stdout | 2023-08-08 06:09:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:25 UTC)" (scheduled at 2023-08-08 06:09:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:09:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:09:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:25 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:09:25,528 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:09:25,732 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:09:26,832 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:09:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:09:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:49.687363+00:00 (in 22.796058 seconds) securityworker stdout | 2023-08-08 06:09:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:56 UTC)" (scheduled at 2023-08-08 06:09:26.890741+00:00) securityworker stdout | 2023-08-08 06:09:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:09:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:09:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:09:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:09:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:09:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:09:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:09:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:09:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:09:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:09:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 4, 26, 896598), 1, 2]) securityworker stdout | 2023-08-08 06:09:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 4, 26, 896598), 1, 2]) securityworker stdout | 2023-08-08 06:09:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:09:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:09:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:09:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:09:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:09:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:09:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:09:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:09:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:09:27,380 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:09:27,417 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} repositoryactioncounter stdout | 2023-08-08 06:09:27,439 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:09:27,500 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:09:27,530 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:09:27,556 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:09:27,559 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:09:28,770 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} chunkcleanupworker stdout | 2023-08-08 06:09:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:09:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:32.293757+00:00 (in 3.418300 seconds) chunkcleanupworker stdout | 2023-08-08 06:09:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:28 UTC)" (scheduled at 2023-08-08 06:09:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:09:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:09:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:28 UTC)" executed successfully builder stdout | 2023-08-08 06:09:28,935 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:09:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:09:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:39.417660+00:00 (in 9.999460 seconds) gcworker stdout | 2023-08-08 06:09:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:39 UTC)" (scheduled at 2023-08-08 06:09:29.417660+00:00) gcworker stdout | 2023-08-08 06:09:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265369418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:09:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:09:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:09:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:09:30,007 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:09:30,902 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:09:30,953 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:09:31,399 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:09:31,920 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:09:32,251 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} chunkcleanupworker stdout | 2023-08-08 06:09:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:09:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:42.293757+00:00 (in 9.999446 seconds) chunkcleanupworker stdout | 2023-08-08 06:09:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:42 UTC)" (scheduled at 2023-08-08 06:09:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:09:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:09:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 32, 294854), True, datetime.datetime(2023, 8, 8, 6, 9, 32, 294854), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:09:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:09:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:09:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:42 UTC)" executed successfully builder stdout | 2023-08-08 06:09:32,441 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:09:32,450 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:09:32,450 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:09:32,455 [55] [DEBUG] [data.database] Disconnecting from database. expiredappspecifictokenworker stdout | 2023-08-08 06:09:32,558 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:09:33,281 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:09:33,284 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:09:33,287 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:09:33,296 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:09:33,299 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:09:33,302 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:09:33,308 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:09:33,313 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:09:33,316 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:09:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:09:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:45.371682+00:00 (in 9.999457 seconds) notificationworker stdout | 2023-08-08 06:09:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:45 UTC)" (scheduled at 2023-08-08 06:09:35.371682+00:00) notificationworker stdout | 2023-08-08 06:09:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:09:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 35, 372599), True, datetime.datetime(2023, 8, 8, 6, 9, 35, 372599), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:09:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:09:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:09:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:09:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:09:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:39.579576+00:00 (in 2.999217 seconds) namespacegcworker stdout | 2023-08-08 06:09:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:36 UTC)" (scheduled at 2023-08-08 06:09:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:09:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:09:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:09:37,180 [197] [DEBUG] [app] Starting request: urn:request:cc06a7e3-5b24-47b0-a067-947a9a7fb031 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:09:37,180 [200] [DEBUG] [app] Starting request: urn:request:e672648a-4d64-4c46-8b95-f2fa6a5c7e4a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:09:37,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:37,186 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:09:37,186 [209] [DEBUG] [app] Starting request: urn:request:5d140bb0-77a2-4ad6-a6e7-8ff4d0f1b731 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:09:37,187 [210] [DEBUG] [app] Starting request: urn:request:00697805-eab6-40eb-857e-11efb6af4285 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:09:37,187 [209] [DEBUG] [app] Ending request: urn:request:5d140bb0-77a2-4ad6-a6e7-8ff4d0f1b731 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5d140bb0-77a2-4ad6-a6e7-8ff4d0f1b731', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:09:37,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:09:37,187 [210] [DEBUG] [app] Ending request: urn:request:00697805-eab6-40eb-857e-11efb6af4285 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:00697805-eab6-40eb-857e-11efb6af4285', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 06:09:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:09:37,188 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:37,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:09:37,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:37,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:37,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:37,192 [197] [DEBUG] [app] Starting request: urn:request:592c3ced-668b-4600-9d36-13ed76fe99f4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:09:37,193 [197] [DEBUG] [app] Ending request: urn:request:592c3ced-668b-4600-9d36-13ed76fe99f4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:592c3ced-668b-4600-9d36-13ed76fe99f4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:09:37,194 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:37,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:09:37,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:09:37,195 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:37,196 [198] [DEBUG] [app] Starting request: urn:request:ff39715f-8134-4dba-ad84-4ff365e26e2c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:09:37,196 [198] [DEBUG] [app] Ending request: urn:request:ff39715f-8134-4dba-ad84-4ff365e26e2c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ff39715f-8134-4dba-ad84-4ff365e26e2c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:09:37,197 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:09:37,197 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 buildlogsarchiver stdout | 2023-08-08 06:09:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:09:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:07.197412+00:00 (in 29.999546 seconds) gunicorn-web stdout | 2023-08-08 06:09:37,198 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:09:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:07 UTC)" (scheduled at 2023-08-08 06:09:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:09:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 9, 37, 198445), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:09:37,200 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:09:37,202 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:09:37,205 [200] [DEBUG] [app] Ending request: urn:request:e672648a-4d64-4c46-8b95-f2fa6a5c7e4a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e672648a-4d64-4c46-8b95-f2fa6a5c7e4a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:09:37,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:09:37,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:09:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:09:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:09:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:09:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 gunicorn-web stdout | 2023-08-08 06:09:37,207 [197] [DEBUG] [app] Ending request: urn:request:cc06a7e3-5b24-47b0-a067-947a9a7fb031 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cc06a7e3-5b24-47b0-a067-947a9a7fb031', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:09:37,207 [197] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:09:37,208 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:09:37,208 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:09:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:09:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.030 47 0.029) buildlogsarchiver stdout | 2023-08-08 06:09:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:09:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:09:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:09:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:49.417660+00:00 (in 9.999475 seconds) gcworker stdout | 2023-08-08 06:09:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:49 UTC)" (scheduled at 2023-08-08 06:09:39.417660+00:00) gcworker stdout | 2023-08-08 06:09:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265379418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:09:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:09:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:09:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:09:39,579 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:09:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:54.393627+00:00 (in 14.813537 seconds) namespacegcworker stdout | 2023-08-08 06:09:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:14:39 UTC)" (scheduled at 2023-08-08 06:09:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:09:39,582 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 9, 39, 581054), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:09:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 9, 39, 581054), True, datetime.datetime(2023, 8, 8, 6, 9, 39, 581054), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:09:39,591 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 9, 39, 581054), True, datetime.datetime(2023, 8, 8, 6, 9, 39, 581054), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 9, 39, 581054), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:09:39,592 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:09:39,592 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:14:39 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:09:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:09:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:46.874674+00:00 (in 4.579903 seconds) chunkcleanupworker stdout | 2023-08-08 06:09:42,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:52 UTC)" (scheduled at 2023-08-08 06:09:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:09:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:09:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 42, 295332), True, datetime.datetime(2023, 8, 8, 6, 9, 42, 295332), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:09:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:09:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:09:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:52 UTC)" executed successfully builder stdout | 2023-08-08 06:09:42,455 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:09:42,464 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:09:42,464 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:09:42,469 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:09:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:09:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:55.371682+00:00 (in 9.999506 seconds) notificationworker stdout | 2023-08-08 06:09:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:55 UTC)" (scheduled at 2023-08-08 06:09:45.371682+00:00) notificationworker stdout | 2023-08-08 06:09:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:09:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 45, 372642), True, datetime.datetime(2023, 8, 8, 6, 9, 45, 372642), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:09:45,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:09:45,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:09:45,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:09:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:09:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:52.293757+00:00 (in 5.418568 seconds) chunkcleanupworker stdout | 2023-08-08 06:09:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:14:46 UTC)" (scheduled at 2023-08-08 06:09:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:09:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 9, 46, 875636), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:09:46,883 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 9, 46, 875636), True, datetime.datetime(2023, 8, 8, 6, 9, 46, 875636), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:09:46,885 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 9, 46, 875636), True, datetime.datetime(2023, 8, 8, 6, 9, 46, 875636), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 9, 46, 875636), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:09:46,886 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:09:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:14:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:09:47,399 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:09:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:25.514341+00:00 (in 38.114775 seconds) exportactionlogsworker stdout | 2023-08-08 06:09:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:47 UTC)" (scheduled at 2023-08-08 06:09:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:09:47,400 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:09:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 47, 400077), True, datetime.datetime(2023, 8, 8, 6, 9, 47, 400077), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:09:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:09:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:09:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:09:48,022 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:09:49,286 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:09:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:09:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:59.417660+00:00 (in 9.999492 seconds) gcworker stdout | 2023-08-08 06:09:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:59 UTC)" (scheduled at 2023-08-08 06:09:49.417660+00:00) gcworker stdout | 2023-08-08 06:09:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265389418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:09:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:09:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:09:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:09:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:09:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:09:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:09:56.890741+00:00 (in 7.202877 seconds) securityworker stdout | 2023-08-08 06:09:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:19 UTC)" (scheduled at 2023-08-08 06:09:49.687363+00:00) securityworker stdout | 2023-08-08 06:09:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:09:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:09:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:09:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:09:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:09:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:09:50,333 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:09:50,789 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:09:52,179 [200] [DEBUG] [app] Starting request: urn:request:1b698a83-3df7-4e5c-842b-51b9865d051b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:09:52,180 [199] [DEBUG] [app] Starting request: urn:request:018d725f-c7f6-4733-ad4a-ede415604deb (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:09:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:09:52,185 [206] [DEBUG] [app] Starting request: urn:request:3186be73-84fe-4d20-8120-ab1099657e63 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:09:52,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:09:52,186 [206] [DEBUG] [app] Ending request: urn:request:3186be73-84fe-4d20-8120-ab1099657e63 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3186be73-84fe-4d20-8120-ab1099657e63', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-registry stdout | 2023-08-08 06:09:52,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:09:52,186 [203] [DEBUG] [app] Starting request: urn:request:52789a6e-a382-4370-8cb3-4129b04d0e5a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:09:52,187 [203] [DEBUG] [app] Ending request: urn:request:52789a6e-a382-4370-8cb3-4129b04d0e5a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:52789a6e-a382-4370-8cb3-4129b04d0e5a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:09:52,188 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:52,188 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:09:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:52,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:09:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:52,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:09:52,192 [200] [DEBUG] [app] Starting request: urn:request:464865a5-35b0-4989-9e51-a4e5a5878f02 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:09:52,192 [200] [DEBUG] [app] Ending request: urn:request:464865a5-35b0-4989-9e51-a4e5a5878f02 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:464865a5-35b0-4989-9e51-a4e5a5878f02', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:09:52,192 [198] [DEBUG] [app] Starting request: urn:request:e5c5bbd0-e6d1-428e-8c8b-fe755a462072 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:09:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:09:52,193 [198] [DEBUG] [app] Ending request: urn:request:e5c5bbd0-e6d1-428e-8c8b-fe755a462072 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e5c5bbd0-e6d1-428e-8c8b-fe755a462072', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:09:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:09:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:09:52,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:09:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:09:52,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:09:52,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:09:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:09:52,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:09:52,204 [200] [DEBUG] [app] Ending request: urn:request:1b698a83-3df7-4e5c-842b-51b9865d051b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1b698a83-3df7-4e5c-842b-51b9865d051b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:09:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:09:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:09:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:09:52,204 [199] [DEBUG] [app] Ending request: urn:request:018d725f-c7f6-4733-ad4a-ede415604deb (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:018d725f-c7f6-4733-ad4a-ede415604deb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:09:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:09:52,204 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:09:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:09:52,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:09:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:09:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:09:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:02.293757+00:00 (in 9.999498 seconds) chunkcleanupworker stdout | 2023-08-08 06:09:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:02 UTC)" (scheduled at 2023-08-08 06:09:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:09:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:09:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 52, 294772), True, datetime.datetime(2023, 8, 8, 6, 9, 52, 294772), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:09:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:09:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:09:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:02 UTC)" executed successfully builder stdout | 2023-08-08 06:09:52,479 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:09:52,488 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:09:52,488 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:09:52,493 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:09:53,538 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:09:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:09:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:13.097375+00:00 (in 18.812777 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:09:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:54 UTC)" (scheduled at 2023-08-08 06:09:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:09:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:09:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:09:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:09:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:36.579841+00:00 (in 42.185716 seconds) namespacegcworker stdout | 2023-08-08 06:09:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:54 UTC)" (scheduled at 2023-08-08 06:09:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:09:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:09:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 54, 394603), True, datetime.datetime(2023, 8, 8, 6, 9, 54, 394603), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:09:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:09:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:09:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:10:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:09:55,016 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:09:55,019 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:09:55,022 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:09:55,357 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} notificationworker stdout | 2023-08-08 06:09:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:09:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:02.178530+00:00 (in 6.806347 seconds) notificationworker stdout | 2023-08-08 06:09:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:05 UTC)" (scheduled at 2023-08-08 06:09:55.371682+00:00) notificationworker stdout | 2023-08-08 06:09:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:09:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 9, 55, 372730), True, datetime.datetime(2023, 8, 8, 6, 9, 55, 372730), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:09:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:09:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:09:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:05 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:09:55,564 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:09:55,769 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:09:56,839 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:09:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:09:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:19.687363+00:00 (in 22.796113 seconds) securityworker stdout | 2023-08-08 06:09:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:26 UTC)" (scheduled at 2023-08-08 06:09:56.890741+00:00) securityworker stdout | 2023-08-08 06:09:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:09:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:09:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:09:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:09:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:09:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:09:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:09:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:09:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:09:56,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:09:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 4, 56, 896385), 1, 2]) securityworker stdout | 2023-08-08 06:09:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 4, 56, 896385), 1, 2]) securityworker stdout | 2023-08-08 06:09:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:09:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:09:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:09:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:09:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:09:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:09:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:09:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:09:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:09:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:26 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:09:57,416 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:09:57,425 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} repositoryactioncounter stdout | 2023-08-08 06:09:57,455 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:09:57,510 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:09:57,543 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:09:57,568 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:09:57,571 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:09:58,777 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:09:58,958 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:09:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:09:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:09.417660+00:00 (in 9.999498 seconds) gcworker stdout | 2023-08-08 06:09:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:09 UTC)" (scheduled at 2023-08-08 06:09:59.417660+00:00) gcworker stdout | 2023-08-08 06:09:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265399418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:09:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:09:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:09:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:10:00,041 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:10:00,931 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:10:00,978 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:10:01,411 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:10:01,936 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} notificationworker stdout | 2023-08-08 06:10:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:10:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:05.371682+00:00 (in 3.192626 seconds) notificationworker stdout | 2023-08-08 06:10:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:02 UTC)" (scheduled at 2023-08-08 06:10:02.178530+00:00) notificationworker stdout | 2023-08-08 06:10:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:10:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:02 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:10:02,260 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} chunkcleanupworker stdout | 2023-08-08 06:10:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:10:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:12.293757+00:00 (in 9.999513 seconds) chunkcleanupworker stdout | 2023-08-08 06:10:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:12 UTC)" (scheduled at 2023-08-08 06:10:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:10:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:10:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 2, 294737), True, datetime.datetime(2023, 8, 8, 6, 10, 2, 294737), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:10:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:10:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:10:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:12 UTC)" executed successfully builder stdout | 2023-08-08 06:10:02,503 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:10:02,512 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:10:02,512 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:10:02,516 [55] [DEBUG] [data.database] Disconnecting from database. expiredappspecifictokenworker stdout | 2023-08-08 06:10:02,577 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:10:03,289 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:10:03,296 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:10:03,298 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:10:03,305 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:10:03,309 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:10:03,312 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:10:03,319 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:10:03,322 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:10:03,327 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:10:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:10:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:09.793658+00:00 (in 5.605682 seconds) repositorygcworker stdout | 2023-08-08 06:10:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:04 UTC)" (scheduled at 2023-08-08 06:10:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:10:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:10:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:10:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:10:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:15.371682+00:00 (in 9.999455 seconds) notificationworker stdout | 2023-08-08 06:10:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:15 UTC)" (scheduled at 2023-08-08 06:10:05.371682+00:00) notificationworker stdout | 2023-08-08 06:10:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:10:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 5, 372761), True, datetime.datetime(2023, 8, 8, 6, 10, 5, 372761), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:10:05,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:10:05,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:10:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:10:07,178 [197] [DEBUG] [app] Starting request: urn:request:17c4ef7c-1bd8-40a9-b8ac-8710c8c927ed (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:10:07,179 [199] [DEBUG] [app] Starting request: urn:request:4399f616-73dc-44a1-97f9-a67af28e09e3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:10:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:10:07,184 [204] [DEBUG] [app] Starting request: urn:request:8f692f70-458b-43a5-8dad-eeb8f9c33b10 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:10:07,184 [204] [DEBUG] [app] Ending request: urn:request:8f692f70-458b-43a5-8dad-eeb8f9c33b10 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8f692f70-458b-43a5-8dad-eeb8f9c33b10', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:10:07,184 [207] [DEBUG] [app] Starting request: urn:request:cb257bf2-3728-4145-99bb-9d9c706b8925 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:10:07,185 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:10:07,185 [207] [DEBUG] [app] Ending request: urn:request:cb257bf2-3728-4145-99bb-9d9c706b8925 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:cb257bf2-3728-4145-99bb-9d9c706b8925', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:10:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:10:07,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:10:07,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:07,186 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:07,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:07,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:07,190 [200] [DEBUG] [app] Starting request: urn:request:114457fa-4a45-45a2-852d-6e9b1e2ad083 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:10:07,190 [198] [DEBUG] [app] Starting request: urn:request:f3e95335-280a-42bb-9166-cacdddba1818 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:10:07,191 [200] [DEBUG] [app] Ending request: urn:request:114457fa-4a45-45a2-852d-6e9b1e2ad083 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:114457fa-4a45-45a2-852d-6e9b1e2ad083', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:10:07,191 [198] [DEBUG] [app] Ending request: urn:request:f3e95335-280a-42bb-9166-cacdddba1818 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f3e95335-280a-42bb-9166-cacdddba1818', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:10:07,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:10:07,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:10:07,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:07,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:07,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:10:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:10:07,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:10:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:10:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:10:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:37.197412+00:00 (in 29.999517 seconds) buildlogsarchiver stdout | 2023-08-08 06:10:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:37 UTC)" (scheduled at 2023-08-08 06:10:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:10:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 10, 7, 198318), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:10:07,202 [199] [DEBUG] [app] Ending request: urn:request:4399f616-73dc-44a1-97f9-a67af28e09e3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4399f616-73dc-44a1-97f9-a67af28e09e3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:10:07,202 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:10:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:10:07,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:10:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:10:07,203 [197] [DEBUG] [app] Ending request: urn:request:17c4ef7c-1bd8-40a9-b8ac-8710c8c927ed (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:17c4ef7c-1bd8-40a9-b8ac-8710c8c927ed', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:10:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:10:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:10:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:10:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:10:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:10:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:10:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:10:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:10:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:10:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:10:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:19.417660+00:00 (in 9.999495 seconds) gcworker stdout | 2023-08-08 06:10:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:19 UTC)" (scheduled at 2023-08-08 06:10:09.417660+00:00) gcworker stdout | 2023-08-08 06:10:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265409418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:10:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:10:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:10:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:10:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:10:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:04.187429+00:00 (in 54.393200 seconds) repositorygcworker stdout | 2023-08-08 06:10:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:09 UTC)" (scheduled at 2023-08-08 06:10:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:10:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:10:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 9, 794806), True, datetime.datetime(2023, 8, 8, 6, 10, 9, 794806), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:10:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:10:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:10:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:10:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:10:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:22.293757+00:00 (in 9.999525 seconds) chunkcleanupworker stdout | 2023-08-08 06:10:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:22 UTC)" (scheduled at 2023-08-08 06:10:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:10:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:10:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 12, 294779), True, datetime.datetime(2023, 8, 8, 6, 10, 12, 294779), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:10:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:10:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:10:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:22 UTC)" executed successfully builder stdout | 2023-08-08 06:10:12,525 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:10:12,534 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:10:12,534 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:10:12,538 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:10:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:10:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:54.284093+00:00 (in 41.186173 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:10:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:13 UTC)" (scheduled at 2023-08-08 06:10:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:10:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:10:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 13, 98495), True, datetime.datetime(2023, 8, 8, 6, 10, 13, 98495), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:10:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:10:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:10:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:10:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:10:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:25.371682+00:00 (in 9.999456 seconds) notificationworker stdout | 2023-08-08 06:10:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:25 UTC)" (scheduled at 2023-08-08 06:10:15.371682+00:00) notificationworker stdout | 2023-08-08 06:10:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:10:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 15, 372729), True, datetime.datetime(2023, 8, 8, 6, 10, 15, 372729), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:10:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:10:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:10:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:10:18,059 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:10:19,315 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:10:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:10:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:29.417660+00:00 (in 9.999441 seconds) gcworker stdout | 2023-08-08 06:10:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:29 UTC)" (scheduled at 2023-08-08 06:10:19.417660+00:00) gcworker stdout | 2023-08-08 06:10:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265419418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:10:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:10:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:10:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:10:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:10:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:26.890741+00:00 (in 7.202858 seconds) securityworker stdout | 2023-08-08 06:10:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:49 UTC)" (scheduled at 2023-08-08 06:10:19.687363+00:00) securityworker stdout | 2023-08-08 06:10:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:10:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:10:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:10:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:10:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:10:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:10:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 5, 19, 691832), 1, 2]) securityworker stdout | 2023-08-08 06:10:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 5, 19, 691832), 1, 2]) securityworker stdout | 2023-08-08 06:10:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:10:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:10:20,368 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:10:20,797 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:10:22,179 [199] [DEBUG] [app] Starting request: urn:request:8cb4fea2-a175-4163-9834-bc43fdccdca8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:10:22,179 [200] [DEBUG] [app] Starting request: urn:request:b17d5acf-a3a1-4618-b9cc-676127db4e0b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:10:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:22,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:10:22,185 [203] [DEBUG] [app] Starting request: urn:request:8a78c84e-9637-4c33-b2f5-03d40cd04372 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:10:22,186 [203] [DEBUG] [app] Ending request: urn:request:8a78c84e-9637-4c33-b2f5-03d40cd04372 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8a78c84e-9637-4c33-b2f5-03d40cd04372', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:10:22,186 [209] [DEBUG] [app] Starting request: urn:request:a170c399-21a1-4956-b4e2-58e9d647d680 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:10:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:10:22,186 [209] [DEBUG] [app] Ending request: urn:request:a170c399-21a1-4956-b4e2-58e9d647d680 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a170c399-21a1-4956-b4e2-58e9d647d680', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:10:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:10:22,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:10:22,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:22,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:22,191 [199] [DEBUG] [app] Starting request: urn:request:b2b99628-3d40-4a6c-838a-22727d32aaa9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:10:22,191 [199] [DEBUG] [app] Ending request: urn:request:b2b99628-3d40-4a6c-838a-22727d32aaa9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b2b99628-3d40-4a6c-838a-22727d32aaa9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:10:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:10:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:10:22,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:22,193 [197] [DEBUG] [app] Starting request: urn:request:2e16bb33-68bc-4695-8876-31dcdcaa6595 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:10:22,194 [197] [DEBUG] [app] Ending request: urn:request:2e16bb33-68bc-4695-8876-31dcdcaa6595 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2e16bb33-68bc-4695-8876-31dcdcaa6595', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:10:22,194 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:10:22,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:22,195 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:10:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:10:22,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:10:22,203 [200] [DEBUG] [app] Ending request: urn:request:b17d5acf-a3a1-4618-b9cc-676127db4e0b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b17d5acf-a3a1-4618-b9cc-676127db4e0b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:10:22,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:10:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:10:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:10:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:10:22,205 [199] [DEBUG] [app] Ending request: urn:request:8cb4fea2-a175-4163-9834-bc43fdccdca8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8cb4fea2-a175-4163-9834-bc43fdccdca8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:10:22,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:10:22,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:10:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:10:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:10:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:10:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:28.874906+00:00 (in 6.580602 seconds) chunkcleanupworker stdout | 2023-08-08 06:10:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:32 UTC)" (scheduled at 2023-08-08 06:10:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:10:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:10:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 22, 294850), True, datetime.datetime(2023, 8, 8, 6, 10, 22, 294850), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:10:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:10:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:10:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:32 UTC)" executed successfully builder stdout | 2023-08-08 06:10:22,548 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:10:22,558 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:10:22,558 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:10:22,563 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:10:23,570 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:10:25,025 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:10:25,030 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:10:25,033 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:10:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:10:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:35.371682+00:00 (in 9.999467 seconds) notificationworker stdout | 2023-08-08 06:10:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:35 UTC)" (scheduled at 2023-08-08 06:10:25.371682+00:00) notificationworker stdout | 2023-08-08 06:10:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:10:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 25, 372751), True, datetime.datetime(2023, 8, 8, 6, 10, 25, 372751), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:10:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:10:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:10:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:35 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:10:25,395 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} exportactionlogsworker stdout | 2023-08-08 06:10:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:10:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:47.398551+00:00 (in 21.883617 seconds) exportactionlogsworker stdout | 2023-08-08 06:10:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:25 UTC)" (scheduled at 2023-08-08 06:10:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:10:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:10:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:25 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:10:25,601 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:10:25,807 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:10:26,862 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:10:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:10:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:49.687363+00:00 (in 22.796109 seconds) securityworker stdout | 2023-08-08 06:10:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:56 UTC)" (scheduled at 2023-08-08 06:10:26.890741+00:00) securityworker stdout | 2023-08-08 06:10:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:10:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:10:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:10:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:10:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:10:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:10:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:10:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:10:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:10:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:10:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 5, 26, 896833), 1, 2]) securityworker stdout | 2023-08-08 06:10:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:26,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 5, 26, 896833), 1, 2]) securityworker stdout | 2023-08-08 06:10:26,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:26,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:26,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:10:26,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:10:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:10:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:10:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:10:26,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:10:26,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:10:26,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:10:56 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:10:27,432 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:10:27,435 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} repositoryactioncounter stdout | 2023-08-08 06:10:27,494 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:10:27,521 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:10:27,556 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:10:27,584 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:10:27,587 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:10:28,797 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} chunkcleanupworker stdout | 2023-08-08 06:10:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:10:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:32.293757+00:00 (in 3.418263 seconds) chunkcleanupworker stdout | 2023-08-08 06:10:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:28 UTC)" (scheduled at 2023-08-08 06:10:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:10:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:10:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:28 UTC)" executed successfully builder stdout | 2023-08-08 06:10:28,986 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:10:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:10:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:39.417660+00:00 (in 9.999496 seconds) gcworker stdout | 2023-08-08 06:10:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:39 UTC)" (scheduled at 2023-08-08 06:10:29.417660+00:00) gcworker stdout | 2023-08-08 06:10:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265429418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:10:29,430 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:10:29,431 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:10:29,431 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:10:30,059 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:10:30,969 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:10:31,015 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:10:31,449 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:10:31,944 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:10:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:10:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:42.293757+00:00 (in 9.999451 seconds) chunkcleanupworker stdout | 2023-08-08 06:10:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:42 UTC)" (scheduled at 2023-08-08 06:10:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:10:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:10:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 32, 294813), True, datetime.datetime(2023, 8, 8, 6, 10, 32, 294813), 0, 'chunk_cleanup/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:10:32,297 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} chunkcleanupworker stdout | 2023-08-08 06:10:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:10:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:10:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:42 UTC)" executed successfully builder stdout | 2023-08-08 06:10:32,564 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:10:32,574 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:10:32,574 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:10:32,579 [55] [DEBUG] [data.database] Disconnecting from database. expiredappspecifictokenworker stdout | 2023-08-08 06:10:32,610 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:10:33,302 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:10:33,307 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:10:33,311 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:10:33,315 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:10:33,322 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:10:33,326 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:10:33,329 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:10:33,332 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:10:33,338 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:10:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:10:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:45.371682+00:00 (in 9.999472 seconds) notificationworker stdout | 2023-08-08 06:10:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:45 UTC)" (scheduled at 2023-08-08 06:10:35.371682+00:00) notificationworker stdout | 2023-08-08 06:10:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:10:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 35, 372594), True, datetime.datetime(2023, 8, 8, 6, 10, 35, 372594), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:10:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:10:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:10:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:10:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:10:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:54.393627+00:00 (in 17.813242 seconds) namespacegcworker stdout | 2023-08-08 06:10:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:36 UTC)" (scheduled at 2023-08-08 06:10:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:10:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:10:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:10:37,179 [197] [DEBUG] [app] Starting request: urn:request:7e8a76fb-f9e6-4d4f-9a36-9361cc418fca (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:10:37,179 [199] [DEBUG] [app] Starting request: urn:request:7b6742b0-49be-4017-8af8-39199e5c8369 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:10:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:37,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:37,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:10:37,185 [203] [DEBUG] [app] Starting request: urn:request:76020aee-fa1e-491c-adc1-56f4ad578586 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:10:37,185 [203] [DEBUG] [app] Ending request: urn:request:76020aee-fa1e-491c-adc1-56f4ad578586 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:76020aee-fa1e-491c-adc1-56f4ad578586', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:10:37,185 [205] [DEBUG] [app] Starting request: urn:request:33720bd7-078c-42d8-8016-f065f7e48348 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:10:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:10:37,186 [205] [DEBUG] [app] Ending request: urn:request:33720bd7-078c-42d8-8016-f065f7e48348 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:33720bd7-078c-42d8-8016-f065f7e48348', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:10:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:10:37,186 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:10:37,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:37,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:37,191 [199] [DEBUG] [app] Starting request: urn:request:50edfb91-3174-447f-a575-87645a77ffdd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:10:37,191 [199] [DEBUG] [app] Ending request: urn:request:50edfb91-3174-447f-a575-87645a77ffdd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:50edfb91-3174-447f-a575-87645a77ffdd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:10:37,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:10:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:10:37,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:10:37,193 [199] [DEBUG] [app] Starting request: urn:request:e31a5c8c-5c47-43b4-9e4f-5e306095a45a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:10:37,193 [199] [DEBUG] [app] Ending request: urn:request:e31a5c8c-5c47-43b4-9e4f-5e306095a45a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e31a5c8c-5c47-43b4-9e4f-5e306095a45a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:10:37,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:10:37,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:37,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:10:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:10:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:10:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:07.197412+00:00 (in 29.999537 seconds) buildlogsarchiver stdout | 2023-08-08 06:10:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:07 UTC)" (scheduled at 2023-08-08 06:10:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:10:37,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:10:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 10, 37, 198406), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:10:37,203 [197] [DEBUG] [app] Ending request: urn:request:7e8a76fb-f9e6-4d4f-9a36-9361cc418fca (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7e8a76fb-f9e6-4d4f-9a36-9361cc418fca', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:10:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:10:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:10:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:10:37,203 [199] [DEBUG] [app] Ending request: urn:request:7b6742b0-49be-4017-8af8-39199e5c8369 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7b6742b0-49be-4017-8af8-39199e5c8369', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:10:37,203 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:10:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:10:37,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:10:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:10:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:10:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:10:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:10:37,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:10:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:10:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:10:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:10:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:49.417660+00:00 (in 9.999082 seconds) gcworker stdout | 2023-08-08 06:10:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:49 UTC)" (scheduled at 2023-08-08 06:10:39.417660+00:00) gcworker stdout | 2023-08-08 06:10:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265439419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:10:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:10:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:10:39,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:10:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:10:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:52.293757+00:00 (in 9.999373 seconds) chunkcleanupworker stdout | 2023-08-08 06:10:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:52 UTC)" (scheduled at 2023-08-08 06:10:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:10:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:10:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 42, 294983), True, datetime.datetime(2023, 8, 8, 6, 10, 42, 294983), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:10:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:10:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:10:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:52 UTC)" executed successfully builder stdout | 2023-08-08 06:10:42,589 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:10:42,598 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:10:42,598 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:10:42,603 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:10:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:10:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:55.371682+00:00 (in 9.999480 seconds) notificationworker stdout | 2023-08-08 06:10:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:55 UTC)" (scheduled at 2023-08-08 06:10:45.371682+00:00) notificationworker stdout | 2023-08-08 06:10:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:10:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 45, 372639), True, datetime.datetime(2023, 8, 8, 6, 10, 45, 372639), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:10:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:10:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:10:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:10:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:10:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:25.514341+00:00 (in 38.115236 seconds) exportactionlogsworker stdout | 2023-08-08 06:10:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:47 UTC)" (scheduled at 2023-08-08 06:10:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:10:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:10:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 47, 399592), True, datetime.datetime(2023, 8, 8, 6, 10, 47, 399592), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:10:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:10:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:10:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:10:48,076 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:10:49,338 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:10:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:10:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:59.417660+00:00 (in 9.999429 seconds) gcworker stdout | 2023-08-08 06:10:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:59 UTC)" (scheduled at 2023-08-08 06:10:49.417660+00:00) gcworker stdout | 2023-08-08 06:10:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265449418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:10:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:10:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:10:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:10:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:10:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:10:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:10:56.890741+00:00 (in 7.202851 seconds) securityworker stdout | 2023-08-08 06:10:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:19 UTC)" (scheduled at 2023-08-08 06:10:49.687363+00:00) securityworker stdout | 2023-08-08 06:10:49,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:10:49,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:10:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:10:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:10:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:10:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:10:50,407 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:10:50,820 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:10:52,179 [200] [DEBUG] [app] Starting request: urn:request:64bebd4b-e398-48dc-9edf-707446f248bf (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:10:52,180 [198] [DEBUG] [app] Starting request: urn:request:4a319494-b836-4c4d-a458-d453484059d1 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:10:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:52,182 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:10:52,185 [210] [DEBUG] [app] Starting request: urn:request:3d38791d-abd9-4905-a693-9b7647374d72 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:10:52,185 [203] [DEBUG] [app] Starting request: urn:request:be1a30b5-1386-4b47-947f-8295dcd99aae (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:10:52,186 [210] [DEBUG] [app] Ending request: urn:request:3d38791d-abd9-4905-a693-9b7647374d72 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3d38791d-abd9-4905-a693-9b7647374d72', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:10:52,186 [203] [DEBUG] [app] Ending request: urn:request:be1a30b5-1386-4b47-947f-8295dcd99aae (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:be1a30b5-1386-4b47-947f-8295dcd99aae', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:10:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:10:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:10:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:10:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:52,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:10:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:52,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:10:52,191 [200] [DEBUG] [app] Starting request: urn:request:cba5f874-df82-4a14-9ff4-b3b3ebe3d363 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:10:52,191 [200] [DEBUG] [app] Ending request: urn:request:cba5f874-df82-4a14-9ff4-b3b3ebe3d363 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:cba5f874-df82-4a14-9ff4-b3b3ebe3d363', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:10:52,192 [199] [DEBUG] [app] Starting request: urn:request:759b1405-78ae-4dd3-b329-17d5476e00e6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:10:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:10:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:52,192 [199] [DEBUG] [app] Ending request: urn:request:759b1405-78ae-4dd3-b329-17d5476e00e6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:759b1405-78ae-4dd3-b329-17d5476e00e6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:10:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:10:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:10:52,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:10:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:10:52,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:10:52,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:10:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:10:52,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:10:52,202 [200] [DEBUG] [app] Ending request: urn:request:64bebd4b-e398-48dc-9edf-707446f248bf (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:64bebd4b-e398-48dc-9edf-707446f248bf', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:10:52,202 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:10:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:10:52,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:10:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:10:52,203 [198] [DEBUG] [app] Ending request: urn:request:4a319494-b836-4c4d-a458-d453484059d1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4a319494-b836-4c4d-a458-d453484059d1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:10:52,203 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:10:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:10:52,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:10:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:10:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:10:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:02.293757+00:00 (in 9.999425 seconds) chunkcleanupworker stdout | 2023-08-08 06:10:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:02 UTC)" (scheduled at 2023-08-08 06:10:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:10:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:10:52,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 52, 295049), True, datetime.datetime(2023, 8, 8, 6, 10, 52, 295049), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:10:52,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:10:52,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:10:52,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:02 UTC)" executed successfully builder stdout | 2023-08-08 06:10:52,603 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:10:52,612 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:10:52,612 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:10:52,616 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:10:53,607 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:10:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:10:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:13.097375+00:00 (in 18.812746 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:10:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:54 UTC)" (scheduled at 2023-08-08 06:10:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:10:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:10:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:10:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:10:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:36.579841+00:00 (in 42.185633 seconds) namespacegcworker stdout | 2023-08-08 06:10:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:54 UTC)" (scheduled at 2023-08-08 06:10:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:10:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:10:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 54, 394850), True, datetime.datetime(2023, 8, 8, 6, 10, 54, 394850), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:10:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:10:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:10:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:11:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:10:55,034 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:10:55,038 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:10:55,041 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:10:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:10:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:02.178530+00:00 (in 6.806223 seconds) notificationworker stdout | 2023-08-08 06:10:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:05 UTC)" (scheduled at 2023-08-08 06:10:55.371682+00:00) notificationworker stdout | 2023-08-08 06:10:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:10:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 10, 55, 373121), True, datetime.datetime(2023, 8, 8, 6, 10, 55, 373121), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:10:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:10:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:10:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:10:55,417 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:10:55,625 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:10:55,844 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} teamsyncworker stdout | 2023-08-08 06:10:56,870 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:10:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:10:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:19.687363+00:00 (in 22.796046 seconds) securityworker stdout | 2023-08-08 06:10:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:26 UTC)" (scheduled at 2023-08-08 06:10:56.890741+00:00) securityworker stdout | 2023-08-08 06:10:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:10:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:10:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:10:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:10:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:10:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:10:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:10:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:10:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:10:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:10:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 5, 56, 896883), 1, 2]) securityworker stdout | 2023-08-08 06:10:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 5, 56, 896883), 1, 2]) securityworker stdout | 2023-08-08 06:10:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:10:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:10:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:10:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:10:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:10:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:10:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:10:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:10:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:10:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:10:57,444 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} queuecleanupworker stdout | 2023-08-08 06:10:57,469 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:10:57,501 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:10:57,533 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:10:57,568 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:10:57,597 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:10:57,600 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:10:58,834 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:10:59,018 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:10:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:10:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:09.417660+00:00 (in 9.999458 seconds) gcworker stdout | 2023-08-08 06:10:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:09 UTC)" (scheduled at 2023-08-08 06:10:59.417660+00:00) gcworker stdout | 2023-08-08 06:10:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265459418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:10:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:10:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:10:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:11:00,073 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:11:00,979 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:11:01,053 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:11:01,487 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:11:01,981 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} notificationworker stdout | 2023-08-08 06:11:02,179 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:11:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:05.371682+00:00 (in 3.192439 seconds) notificationworker stdout | 2023-08-08 06:11:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:02 UTC)" (scheduled at 2023-08-08 06:11:02.178530+00:00) notificationworker stdout | 2023-08-08 06:11:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:11:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:11:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:11:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:12.293757+00:00 (in 9.999437 seconds) chunkcleanupworker stdout | 2023-08-08 06:11:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:12 UTC)" (scheduled at 2023-08-08 06:11:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:11:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:11:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 2, 294899), True, datetime.datetime(2023, 8, 8, 6, 11, 2, 294899), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:11:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:11:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:11:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:12 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:11:02,336 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} builder stdout | 2023-08-08 06:11:02,618 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue expiredappspecifictokenworker stdout | 2023-08-08 06:11:02,625 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:11:02,627 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:11:02,627 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:11:02,631 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:11:03,315 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:11:03,318 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:11:03,321 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:11:03,324 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:11:03,335 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:11:03,338 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:11:03,341 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:11:03,343 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:11:03,349 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:11:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:11:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:09.793658+00:00 (in 5.605741 seconds) repositorygcworker stdout | 2023-08-08 06:11:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:04 UTC)" (scheduled at 2023-08-08 06:11:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:11:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:11:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:11:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:11:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:15.371682+00:00 (in 9.999493 seconds) notificationworker stdout | 2023-08-08 06:11:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:15 UTC)" (scheduled at 2023-08-08 06:11:05.371682+00:00) notificationworker stdout | 2023-08-08 06:11:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:11:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 5, 372882), True, datetime.datetime(2023, 8, 8, 6, 11, 5, 372882), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:11:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:11:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:11:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:11:07,178 [197] [DEBUG] [app] Starting request: urn:request:e2110b49-1318-422b-988c-4fa41b8b0af4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:11:07,180 [200] [DEBUG] [app] Starting request: urn:request:525660f9-6569-468e-a1c4-c973ae386d3f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:11:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:11:07,184 [204] [DEBUG] [app] Starting request: urn:request:4a250c43-af3e-4a14-b114-f39e8bc96e4a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:11:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:11:07,184 [204] [DEBUG] [app] Ending request: urn:request:4a250c43-af3e-4a14-b114-f39e8bc96e4a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4a250c43-af3e-4a14-b114-f39e8bc96e4a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:11:07,185 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:11:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:11:07,185 [210] [DEBUG] [app] Starting request: urn:request:a2ab81c5-72e2-4fa6-89b5-3525748fa43a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:11:07,186 [210] [DEBUG] [app] Ending request: urn:request:a2ab81c5-72e2-4fa6-89b5-3525748fa43a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a2ab81c5-72e2-4fa6-89b5-3525748fa43a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:11:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:11:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:07,191 [198] [DEBUG] [app] Starting request: urn:request:f40f4dad-1dbd-469a-aa56-afa1288c0aef (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:11:07,191 [198] [DEBUG] [app] Ending request: urn:request:f40f4dad-1dbd-469a-aa56-afa1288c0aef (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f40f4dad-1dbd-469a-aa56-afa1288c0aef', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:11:07,191 [199] [DEBUG] [app] Starting request: urn:request:fead4c2b-794f-4393-be18-90b37387c93a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:11:07,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:11:07,192 [199] [DEBUG] [app] Ending request: urn:request:fead4c2b-794f-4393-be18-90b37387c93a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fead4c2b-794f-4393-be18-90b37387c93a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:11:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:11:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:11:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:11:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:11:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:11:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:11:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:37.197412+00:00 (in 29.999541 seconds) buildlogsarchiver stdout | 2023-08-08 06:11:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:37 UTC)" (scheduled at 2023-08-08 06:11:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:11:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 11, 7, 198360), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:11:07,203 [200] [DEBUG] [app] Ending request: urn:request:525660f9-6569-468e-a1c4-c973ae386d3f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:525660f9-6569-468e-a1c4-c973ae386d3f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:11:07,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:11:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:11:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:11:07,203 [197] [DEBUG] [app] Ending request: urn:request:e2110b49-1318-422b-988c-4fa41b8b0af4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e2110b49-1318-422b-988c-4fa41b8b0af4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:11:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:11:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:11:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:11:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:11:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:11:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:11:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:11:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:11:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:11:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:19.417660+00:00 (in 9.999345 seconds) gcworker stdout | 2023-08-08 06:11:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:19 UTC)" (scheduled at 2023-08-08 06:11:09.417660+00:00) gcworker stdout | 2023-08-08 06:11:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265469418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:11:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:11:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:11:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:11:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:11:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:04.187429+00:00 (in 54.393231 seconds) repositorygcworker stdout | 2023-08-08 06:11:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:09 UTC)" (scheduled at 2023-08-08 06:11:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:11:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:11:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 9, 794782), True, datetime.datetime(2023, 8, 8, 6, 11, 9, 794782), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:11:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:11:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:11:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:11:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:11:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:22.293757+00:00 (in 9.999489 seconds) chunkcleanupworker stdout | 2023-08-08 06:11:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:22 UTC)" (scheduled at 2023-08-08 06:11:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:11:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:11:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 12, 294830), True, datetime.datetime(2023, 8, 8, 6, 11, 12, 294830), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:11:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:11:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:11:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:22 UTC)" executed successfully builder stdout | 2023-08-08 06:11:12,641 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:11:12,650 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:11:12,650 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:11:12,654 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:11:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:11:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:31.283850+00:00 (in 18.185900 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:11:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:13 UTC)" (scheduled at 2023-08-08 06:11:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:11:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:11:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 13, 98484), True, datetime.datetime(2023, 8, 8, 6, 11, 13, 98484), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:11:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:11:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:11:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:11:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:11:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:25.371682+00:00 (in 9.999460 seconds) notificationworker stdout | 2023-08-08 06:11:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:25 UTC)" (scheduled at 2023-08-08 06:11:15.371682+00:00) notificationworker stdout | 2023-08-08 06:11:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:11:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 15, 372750), True, datetime.datetime(2023, 8, 8, 6, 11, 15, 372750), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:11:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:11:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:11:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:11:18,100 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:11:19,374 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:11:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:11:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:29.417660+00:00 (in 9.999444 seconds) gcworker stdout | 2023-08-08 06:11:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:29 UTC)" (scheduled at 2023-08-08 06:11:19.417660+00:00) gcworker stdout | 2023-08-08 06:11:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265479418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:11:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:11:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:11:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:11:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:11:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:26.890741+00:00 (in 7.202751 seconds) securityworker stdout | 2023-08-08 06:11:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:49 UTC)" (scheduled at 2023-08-08 06:11:19.687363+00:00) securityworker stdout | 2023-08-08 06:11:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:11:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:11:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:11:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:11:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:11:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:11:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 6, 19, 691971), 1, 2]) securityworker stdout | 2023-08-08 06:11:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 6, 19, 691971), 1, 2]) securityworker stdout | 2023-08-08 06:11:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:11:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:11:20,418 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:11:20,856 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:11:22,179 [199] [DEBUG] [app] Starting request: urn:request:bf2befe9-30a5-4255-a86a-a1c91026fb64 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:11:22,179 [200] [DEBUG] [app] Starting request: urn:request:b15ba98d-e904-47fb-a2cb-480c151d8d28 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:11:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:11:22,185 [208] [DEBUG] [app] Starting request: urn:request:6c371027-039c-4ef6-8c55-99c80edfe9e7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:11:22,185 [203] [DEBUG] [app] Starting request: urn:request:48058a18-0bd0-4691-a48e-6da1d46e50bd (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:11:22,186 [203] [DEBUG] [app] Ending request: urn:request:48058a18-0bd0-4691-a48e-6da1d46e50bd (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:48058a18-0bd0-4691-a48e-6da1d46e50bd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:11:22,186 [208] [DEBUG] [app] Ending request: urn:request:6c371027-039c-4ef6-8c55-99c80edfe9e7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6c371027-039c-4ef6-8c55-99c80edfe9e7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:11:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:11:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 06:11:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:22,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:22,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:22,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:22,191 [200] [DEBUG] [app] Starting request: urn:request:f6bfcc90-a891-49df-a622-c47fcfff5df1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:11:22,192 [200] [DEBUG] [app] Ending request: urn:request:f6bfcc90-a891-49df-a622-c47fcfff5df1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f6bfcc90-a891-49df-a622-c47fcfff5df1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:11:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:11:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:22,192 [198] [DEBUG] [app] Starting request: urn:request:de38fbfc-e557-46c5-9a19-431f1556ade4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:11:22,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:11:22,193 [198] [DEBUG] [app] Ending request: urn:request:de38fbfc-e557-46c5-9a19-431f1556ade4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:de38fbfc-e557-46c5-9a19-431f1556ade4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:11:22,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:11:22,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:22,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:11:22,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:11:22,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:11:22,203 [200] [DEBUG] [app] Ending request: urn:request:b15ba98d-e904-47fb-a2cb-480c151d8d28 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b15ba98d-e904-47fb-a2cb-480c151d8d28', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:11:22,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:22,203 [199] [DEBUG] [app] Ending request: urn:request:bf2befe9-30a5-4255-a86a-a1c91026fb64 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bf2befe9-30a5-4255-a86a-a1c91026fb64', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:11:22,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:11:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:11:22,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:11:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:11:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:11:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:11:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:11:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:28.874906+00:00 (in 6.580658 seconds) chunkcleanupworker stdout | 2023-08-08 06:11:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:32 UTC)" (scheduled at 2023-08-08 06:11:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:11:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:11:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 22, 294793), True, datetime.datetime(2023, 8, 8, 6, 11, 22, 294793), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:11:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:11:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:11:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:32 UTC)" executed successfully builder stdout | 2023-08-08 06:11:22,659 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:11:22,668 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:11:22,668 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:11:22,672 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:11:23,644 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:11:25,042 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:11:25,046 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:11:25,048 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:11:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:11:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:35.371682+00:00 (in 9.999466 seconds) notificationworker stdout | 2023-08-08 06:11:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:35 UTC)" (scheduled at 2023-08-08 06:11:25.371682+00:00) notificationworker stdout | 2023-08-08 06:11:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:11:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 25, 372738), True, datetime.datetime(2023, 8, 8, 6, 11, 25, 372738), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:11:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:11:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:11:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:35 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:11:25,424 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} exportactionlogsworker stdout | 2023-08-08 06:11:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:11:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:47.398551+00:00 (in 21.883647 seconds) exportactionlogsworker stdout | 2023-08-08 06:11:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:25 UTC)" (scheduled at 2023-08-08 06:11:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:11:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:11:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:25 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:11:25,655 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:11:25,882 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:11:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:11:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:49.687363+00:00 (in 22.796143 seconds) securityworker stdout | 2023-08-08 06:11:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:56 UTC)" (scheduled at 2023-08-08 06:11:26.890741+00:00) securityworker stdout | 2023-08-08 06:11:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:11:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:11:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:11:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:11:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:11:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:11:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:11:26,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:11:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:11:26,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:11:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 6, 26, 896120), 1, 2]) securityworker stdout | 2023-08-08 06:11:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 6, 26, 896120), 1, 2]) teamsyncworker stdout | 2023-08-08 06:11:26,907 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} securityworker stdout | 2023-08-08 06:11:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:11:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:11:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:11:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:11:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:11:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:11:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:11:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:11:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:11:27,453 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} queuecleanupworker stdout | 2023-08-08 06:11:27,477 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:11:27,538 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:11:27,542 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:11:27,583 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:11:27,609 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:11:27,612 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:11:28,849 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} chunkcleanupworker stdout | 2023-08-08 06:11:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:11:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:32.293757+00:00 (in 3.418346 seconds) chunkcleanupworker stdout | 2023-08-08 06:11:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:28 UTC)" (scheduled at 2023-08-08 06:11:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:11:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:11:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:28 UTC)" executed successfully builder stdout | 2023-08-08 06:11:29,032 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:11:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:11:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:39.417660+00:00 (in 9.999473 seconds) gcworker stdout | 2023-08-08 06:11:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:39 UTC)" (scheduled at 2023-08-08 06:11:29.417660+00:00) gcworker stdout | 2023-08-08 06:11:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265489418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:11:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:11:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:11:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:11:30,111 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:11:30,991 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:11:31,077 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} securityscanningnotificationworker stdout | 2023-08-08 06:11:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:11:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:54.284093+00:00 (in 22.999729 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:11:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:16:31 UTC)" (scheduled at 2023-08-08 06:11:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:11:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 11, 31, 284857), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:11:31,293 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 11, 31, 284857), True, datetime.datetime(2023, 8, 8, 6, 11, 31, 284857), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:11:31,296 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 11, 31, 284857), True, datetime.datetime(2023, 8, 8, 6, 11, 31, 284857), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 11, 31, 284857), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:11:31,297 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:11:31,297 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:16:31 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:11:31,525 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:11:32,004 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:11:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:11:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:42.293757+00:00 (in 9.999434 seconds) chunkcleanupworker stdout | 2023-08-08 06:11:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:42 UTC)" (scheduled at 2023-08-08 06:11:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:11:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:11:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 32, 294879), True, datetime.datetime(2023, 8, 8, 6, 11, 32, 294879), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:11:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:11:32,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:11:32,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:42 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:11:32,373 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:11:32,656 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:11:32,682 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:11:32,692 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:11:32,692 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:11:32,696 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:33,164 [199] [DEBUG] [app] Starting request: urn:request:fadf5f8f-5fac-4c92-8223-e7da25bd07c1 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:11:33,164 [197] [DEBUG] [app] Starting request: urn:request:bd8b8fff-49e8-4fa2-807c-ccf4ccdb34e3 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:11:33,164 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:11:33,164 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:11:33,164 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:11:33,165 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:11:33,165 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:11:33,165 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:11:33,171 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:11:33,171 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:11:33,172 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:11:33,172 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:11:33,172 [199] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:11:33,172 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:11:33,172 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:11:33,172 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:11:33,172 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:11:33,172 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:11:33,172 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:11:33,172 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:11:33,172 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:11:33,172 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:11:33,172 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:11:33,172 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:11:33,172 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 23, 33, 172517), datetime.datetime(2023, 8, 8, 6, 11, 33, 172566)]) gunicorn-web stdout | 2023-08-08 06:11:33,174 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:11:33,174 [199] [DEBUG] [app] Ending request: urn:request:fadf5f8f-5fac-4c92-8223-e7da25bd07c1 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:fadf5f8f-5fac-4c92-8223-e7da25bd07c1', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:11:33,175 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:11:33 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 62 0.012) gunicorn-web stdout | 2023-08-08 06:11:33,175 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:11:33 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:11:33,178 [197] [DEBUG] [app] Ending request: urn:request:bd8b8fff-49e8-4fa2-807c-ccf4ccdb34e3 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:bd8b8fff-49e8-4fa2-807c-ccf4ccdb34e3', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:11:33,179 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:11:33 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 871 0.017) gunicorn-web stdout | 2023-08-08 06:11:33,179 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:11:33 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-registry stdout | 2023-08-08 06:11:33,327 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:11:33,330 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:11:33,333 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:11:33,336 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:11:33,345 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:11:33,348 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:11:33,351 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:11:33,353 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:11:33,361 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:11:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:11:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:45.371682+00:00 (in 9.999468 seconds) notificationworker stdout | 2023-08-08 06:11:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:45 UTC)" (scheduled at 2023-08-08 06:11:35.371682+00:00) notificationworker stdout | 2023-08-08 06:11:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:11:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 35, 372766), True, datetime.datetime(2023, 8, 8, 6, 11, 35, 372766), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:11:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:11:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:11:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:11:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:11:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:54.393627+00:00 (in 17.813235 seconds) namespacegcworker stdout | 2023-08-08 06:11:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:36 UTC)" (scheduled at 2023-08-08 06:11:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:11:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:11:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:11:37,178 [197] [DEBUG] [app] Starting request: urn:request:0cfb8e32-a154-44b0-b29c-712ee2f414c2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:11:37,179 [199] [DEBUG] [app] Starting request: urn:request:e2c4c87c-5fc2-4e0f-8e3f-748e840e6459 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:11:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:11:37,184 [209] [DEBUG] [app] Starting request: urn:request:19029726-506c-4475-bd99-ce27b7ad391f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:11:37,185 [203] [DEBUG] [app] Starting request: urn:request:b98194d9-78a5-4350-ae65-f9b05e3bb7ff (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:11:37,185 [209] [DEBUG] [app] Ending request: urn:request:19029726-506c-4475-bd99-ce27b7ad391f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:19029726-506c-4475-bd99-ce27b7ad391f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:11:37,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:11:37,185 [203] [DEBUG] [app] Ending request: urn:request:b98194d9-78a5-4350-ae65-f9b05e3bb7ff (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b98194d9-78a5-4350-ae65-f9b05e3bb7ff', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:11:37,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:11:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:11:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:37,190 [200] [DEBUG] [app] Starting request: urn:request:eebd9d41-c203-403f-8c62-6802b06bef75 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:11:37,191 [200] [DEBUG] [app] Ending request: urn:request:eebd9d41-c203-403f-8c62-6802b06bef75 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:eebd9d41-c203-403f-8c62-6802b06bef75', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:11:37,191 [197] [DEBUG] [app] Starting request: urn:request:b66a0ad7-1b20-4a55-ba87-0e80ba63883d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:11:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:11:37,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:37,191 [197] [DEBUG] [app] Ending request: urn:request:b66a0ad7-1b20-4a55-ba87-0e80ba63883d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b66a0ad7-1b20-4a55-ba87-0e80ba63883d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:11:37,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:11:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:11:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:11:37,196 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:11:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:11:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:11:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:07.197412+00:00 (in 29.999519 seconds) buildlogsarchiver stdout | 2023-08-08 06:11:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:07 UTC)" (scheduled at 2023-08-08 06:11:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:11:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 11, 37, 198225), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:11:37,201 [199] [DEBUG] [app] Ending request: urn:request:e2c4c87c-5fc2-4e0f-8e3f-748e840e6459 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e2c4c87c-5fc2-4e0f-8e3f-748e840e6459', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:11:37,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:37,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:11:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:11:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:11:37,203 [197] [DEBUG] [app] Ending request: urn:request:0cfb8e32-a154-44b0-b29c-712ee2f414c2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0cfb8e32-a154-44b0-b29c-712ee2f414c2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:11:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:11:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:11:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:11:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:11:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:11:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:11:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:11:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:11:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:11:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:49.417660+00:00 (in 9.999489 seconds) gcworker stdout | 2023-08-08 06:11:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:49 UTC)" (scheduled at 2023-08-08 06:11:39.417660+00:00) gcworker stdout | 2023-08-08 06:11:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265499418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:11:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:11:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:11:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:49 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:11:41,472 [54] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2023-08-08 06:11:41,472 [54] [INFO] [apscheduler.executors.default] Running job "BlobUploadCleanupWorker._try_cleanup_uploads (trigger: interval[1:00:00], next run at: 2023-08-08 06:11:41 UTC)" (scheduled at 2023-08-08 06:11:41.471663+00:00) blobuploadcleanupworker stdout | 2023-08-08 06:11:41,472 [54] [DEBUG] [util.locking] Acquiring global lock BLOB_CLEANUP blobuploadcleanupworker stdout | 2023-08-08 06:11:41,473 [54] [DEBUG] [redis_lock.acquire] Getting 'lock:BLOB_CLEANUP' ... blobuploadcleanupworker stdout | 2023-08-08 06:11:41,473 [54] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 07:11:41.471663+00:00 (in 3599.999496 seconds) blobuploadcleanupworker stdout | 2023-08-08 06:11:41,473 [54] [INFO] [redis_lock.acquire] Got lock for 'lock:BLOB_CLEANUP'. blobuploadcleanupworker stdout | 2023-08-08 06:11:41,473 [54] [DEBUG] [util.locking] Acquired lock BLOB_CLEANUP blobuploadcleanupworker stdout | 2023-08-08 06:11:41,473 [54] [DEBUG] [__main__] Performing blob upload cleanup blobuploadcleanupworker stdout | 2023-08-08 06:11:41,474 [54] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."created" <= %s) LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 6, 6, 11, 41, 473863), 1, 0]) blobuploadcleanupworker stdout | 2023-08-08 06:11:41,481 [54] [DEBUG] [__main__] No additional stale blob uploads found blobuploadcleanupworker stdout | 2023-08-08 06:11:41,481 [54] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:11:41,481 [54] [DEBUG] [__main__] Cleaning partial uploads not applicable to storage location "default" blobuploadcleanupworker stdout | 2023-08-08 06:11:41,481 [54] [DEBUG] [util.locking] Releasing lock BLOB_CLEANUP blobuploadcleanupworker stdout | 2023-08-08 06:11:41,481 [54] [DEBUG] [redis_lock.release] Releasing 'lock:BLOB_CLEANUP'. blobuploadcleanupworker stdout | 2023-08-08 06:11:41,482 [54] [DEBUG] [util.locking] Released lock BLOB_CLEANUP blobuploadcleanupworker stdout | 2023-08-08 06:11:41,482 [54] [INFO] [apscheduler.executors.default] Job "BlobUploadCleanupWorker._try_cleanup_uploads (trigger: interval[1:00:00], next run at: 2023-08-08 07:11:41 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:11:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:11:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:52.293757+00:00 (in 9.999410 seconds) chunkcleanupworker stdout | 2023-08-08 06:11:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:52 UTC)" (scheduled at 2023-08-08 06:11:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:11:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:11:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 42, 294772), True, datetime.datetime(2023, 8, 8, 6, 11, 42, 294772), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:11:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:11:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:11:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:52 UTC)" executed successfully builder stdout | 2023-08-08 06:11:42,702 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:11:42,711 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:11:42,711 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:11:42,715 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:11:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:11:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:55.371682+00:00 (in 9.999422 seconds) notificationworker stdout | 2023-08-08 06:11:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:55 UTC)" (scheduled at 2023-08-08 06:11:45.371682+00:00) notificationworker stdout | 2023-08-08 06:11:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:11:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 45, 372797), True, datetime.datetime(2023, 8, 8, 6, 11, 45, 372797), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:11:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:11:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:11:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:11:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:11:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:25.514341+00:00 (in 38.115224 seconds) exportactionlogsworker stdout | 2023-08-08 06:11:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:47 UTC)" (scheduled at 2023-08-08 06:11:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:11:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:11:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 47, 399631), True, datetime.datetime(2023, 8, 8, 6, 11, 47, 399631), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:11:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:11:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:11:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:11:48,137 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:11:49,411 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gcworker stdout | 2023-08-08 06:11:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:11:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:59.417660+00:00 (in 9.999461 seconds) gcworker stdout | 2023-08-08 06:11:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:59 UTC)" (scheduled at 2023-08-08 06:11:49.417660+00:00) gcworker stdout | 2023-08-08 06:11:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265509418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:11:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:11:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:11:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:11:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:11:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:11:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:11:56.890741+00:00 (in 7.202715 seconds) securityworker stdout | 2023-08-08 06:11:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:19 UTC)" (scheduled at 2023-08-08 06:11:49.687363+00:00) securityworker stdout | 2023-08-08 06:11:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:11:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:11:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:11:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:11:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:11:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:11:50,440 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:11:50,893 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:11:52,179 [200] [DEBUG] [app] Starting request: urn:request:92925f7c-693c-49e9-a9eb-2c12fe92e677 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:11:52,179 [197] [DEBUG] [app] Starting request: urn:request:decb6e16-5fa9-4f8c-92cb-c2fba8491e2d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:11:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:11:52,184 [207] [DEBUG] [app] Starting request: urn:request:a33902ea-ee5c-4a77-abcd-8bef7efd286e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:11:52,184 [209] [DEBUG] [app] Starting request: urn:request:578d0b1b-20a0-4dee-8b92-e884c5c8f532 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:11:52,185 [207] [DEBUG] [app] Ending request: urn:request:a33902ea-ee5c-4a77-abcd-8bef7efd286e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a33902ea-ee5c-4a77-abcd-8bef7efd286e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:11:52,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:11:52,185 [209] [DEBUG] [app] Ending request: urn:request:578d0b1b-20a0-4dee-8b92-e884c5c8f532 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:578d0b1b-20a0-4dee-8b92-e884c5c8f532', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:11:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:11:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:11:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:11:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:52,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:11:52,190 [198] [DEBUG] [app] Starting request: urn:request:0524b48f-a22f-4519-8091-af7f344ae70a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:11:52,190 [199] [DEBUG] [app] Starting request: urn:request:25286030-1aad-452f-ba86-d7bb9f6eb554 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:11:52,191 [198] [DEBUG] [app] Ending request: urn:request:0524b48f-a22f-4519-8091-af7f344ae70a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0524b48f-a22f-4519-8091-af7f344ae70a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:11:52,191 [199] [DEBUG] [app] Ending request: urn:request:25286030-1aad-452f-ba86-d7bb9f6eb554 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:25286030-1aad-452f-ba86-d7bb9f6eb554', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:11:52,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:11:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:11:52,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:11:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:11:52,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:11:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:11:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:11:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:11:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:11:52,202 [197] [DEBUG] [app] Ending request: urn:request:decb6e16-5fa9-4f8c-92cb-c2fba8491e2d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:decb6e16-5fa9-4f8c-92cb-c2fba8491e2d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:11:52,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:52,202 [200] [DEBUG] [app] Ending request: urn:request:92925f7c-693c-49e9-a9eb-2c12fe92e677 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:92925f7c-693c-49e9-a9eb-2c12fe92e677', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:11:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:11:52,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:11:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:11:52,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:11:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:11:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:11:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:11:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:11:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:02.293757+00:00 (in 9.999492 seconds) chunkcleanupworker stdout | 2023-08-08 06:11:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:02 UTC)" (scheduled at 2023-08-08 06:11:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:11:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:11:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 52, 294795), True, datetime.datetime(2023, 8, 8, 6, 11, 52, 294795), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:11:52,301 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:11:52,301 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:11:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:02 UTC)" executed successfully builder stdout | 2023-08-08 06:11:52,725 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:11:52,734 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:11:52,734 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:11:52,739 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:11:53,680 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:11:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:11:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:13.097375+00:00 (in 18.812688 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:11:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:54 UTC)" (scheduled at 2023-08-08 06:11:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:11:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:11:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:11:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:11:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:36.579841+00:00 (in 42.185610 seconds) namespacegcworker stdout | 2023-08-08 06:11:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:54 UTC)" (scheduled at 2023-08-08 06:11:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:11:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:11:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 54, 395022), True, datetime.datetime(2023, 8, 8, 6, 11, 54, 395022), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:11:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:11:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:11:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:12:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:11:55,052 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:11:55,056 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:11:55,059 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:11:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:11:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:02.178530+00:00 (in 6.806288 seconds) notificationworker stdout | 2023-08-08 06:11:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:05 UTC)" (scheduled at 2023-08-08 06:11:55.371682+00:00) notificationworker stdout | 2023-08-08 06:11:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:11:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 11, 55, 372733), True, datetime.datetime(2023, 8, 8, 6, 11, 55, 372733), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:11:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:11:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:11:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:11:55,434 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:11:55,692 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:11:55,919 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:11:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:11:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:19.687363+00:00 (in 22.796085 seconds) securityworker stdout | 2023-08-08 06:11:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:26 UTC)" (scheduled at 2023-08-08 06:11:56.890741+00:00) securityworker stdout | 2023-08-08 06:11:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:11:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:11:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:11:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:11:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:11:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:11:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:11:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:11:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:11:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:11:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 6, 56, 896699), 1, 2]) securityworker stdout | 2023-08-08 06:11:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 6, 56, 896699), 1, 2]) securityworker stdout | 2023-08-08 06:11:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:11:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:11:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:11:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:11:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:11:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:11:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:11:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:11:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:11:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:11:56,936 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:11:57,464 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} queuecleanupworker stdout | 2023-08-08 06:11:57,514 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:11:57,544 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:11:57,551 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:11:57,595 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:11:57,623 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:11:57,626 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:11:58,886 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:11:59,071 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:11:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:11:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:09.417660+00:00 (in 9.999447 seconds) gcworker stdout | 2023-08-08 06:11:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:09 UTC)" (scheduled at 2023-08-08 06:11:59.417660+00:00) gcworker stdout | 2023-08-08 06:11:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265519418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:11:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:11:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:11:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:12:00,148 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:12:00,999 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:12:01,104 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:12:01,562 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:12:02,028 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} notificationworker stdout | 2023-08-08 06:12:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:12:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:05.371682+00:00 (in 3.192639 seconds) notificationworker stdout | 2023-08-08 06:12:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:02 UTC)" (scheduled at 2023-08-08 06:12:02.178530+00:00) notificationworker stdout | 2023-08-08 06:12:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:12:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:12:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:12:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:12.293757+00:00 (in 9.999389 seconds) chunkcleanupworker stdout | 2023-08-08 06:12:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:12 UTC)" (scheduled at 2023-08-08 06:12:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:12:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:12:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 2, 294852), True, datetime.datetime(2023, 8, 8, 6, 12, 2, 294852), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:12:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:12:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:12:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:12 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:12:02,410 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:12:02,677 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:12:02,749 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:12:02,758 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:12:02,758 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:12:02,762 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:12:03,335 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:12:03,340 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:12:03,344 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:12:03,347 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:12:03,354 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:12:03,359 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:12:03,362 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:12:03,365 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:12:03,371 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:12:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:12:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:09.793658+00:00 (in 5.605672 seconds) repositorygcworker stdout | 2023-08-08 06:12:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:04 UTC)" (scheduled at 2023-08-08 06:12:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:12:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:12:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:12:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:12:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:15.371682+00:00 (in 9.999273 seconds) notificationworker stdout | 2023-08-08 06:12:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:15 UTC)" (scheduled at 2023-08-08 06:12:05.371682+00:00) notificationworker stdout | 2023-08-08 06:12:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:12:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 5, 372899), True, datetime.datetime(2023, 8, 8, 6, 12, 5, 372899), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:12:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:12:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:12:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:12:07,179 [200] [DEBUG] [app] Starting request: urn:request:3cb488e1-d6dd-4542-aee0-5c39287eb7b6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:12:07,180 [197] [DEBUG] [app] Starting request: urn:request:14d4a8ad-25e6-4f93-aee7-11653dd00b2d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:12:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:07,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:07,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:12:07,185 [209] [DEBUG] [app] Starting request: urn:request:bb9dc418-2d54-4626-a8f4-6af0efc1ce93 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:12:07,185 [209] [DEBUG] [app] Ending request: urn:request:bb9dc418-2d54-4626-a8f4-6af0efc1ce93 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:bb9dc418-2d54-4626-a8f4-6af0efc1ce93', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:12:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:12:07,186 [204] [DEBUG] [app] Starting request: urn:request:77d45ae9-ce80-49cf-a3a1-de23df58b579 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:12:07,186 [204] [DEBUG] [app] Ending request: urn:request:77d45ae9-ce80-49cf-a3a1-de23df58b579 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:77d45ae9-ce80-49cf-a3a1-de23df58b579', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:12:07,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:12:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:07,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:07,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:07,190 [198] [DEBUG] [app] Starting request: urn:request:f4af2865-52fa-4cc0-bd80-d2d84581783e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:07,191 [198] [DEBUG] [app] Ending request: urn:request:f4af2865-52fa-4cc0-bd80-d2d84581783e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f4af2865-52fa-4cc0-bd80-d2d84581783e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:12:07,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:12:07,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:12:07,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:12:07,192 [199] [DEBUG] [app] Starting request: urn:request:dc9e5c2b-0002-4c2a-91ab-22692b327534 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:07,193 [199] [DEBUG] [app] Ending request: urn:request:dc9e5c2b-0002-4c2a-91ab-22692b327534 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:dc9e5c2b-0002-4c2a-91ab-22692b327534', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:12:07,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:12:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:07,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:12:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:12:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:12:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:37.197412+00:00 (in 29.999520 seconds) buildlogsarchiver stdout | 2023-08-08 06:12:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:37 UTC)" (scheduled at 2023-08-08 06:12:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:12:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 12, 7, 198230), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:12:07,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:12:07,203 [200] [DEBUG] [app] Ending request: urn:request:3cb488e1-d6dd-4542-aee0-5c39287eb7b6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3cb488e1-d6dd-4542-aee0-5c39287eb7b6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:12:07,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:12:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:12:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:12:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:12:07,205 [197] [DEBUG] [app] Ending request: urn:request:14d4a8ad-25e6-4f93-aee7-11653dd00b2d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:14d4a8ad-25e6-4f93-aee7-11653dd00b2d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:12:07,206 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:12:07,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:12:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:12:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:12:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:12:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:12:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:12:07,212 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:12:07,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:12:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:12:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:19.417660+00:00 (in 9.999451 seconds) gcworker stdout | 2023-08-08 06:12:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:19 UTC)" (scheduled at 2023-08-08 06:12:09.417660+00:00) gcworker stdout | 2023-08-08 06:12:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265529418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:12:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:12:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:12:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:12:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:12:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:04.187429+00:00 (in 54.393179 seconds) repositorygcworker stdout | 2023-08-08 06:12:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:09 UTC)" (scheduled at 2023-08-08 06:12:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:12:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:12:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 9, 794815), True, datetime.datetime(2023, 8, 8, 6, 12, 9, 794815), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:12:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:12:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:12:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:12:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:12:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:22.293757+00:00 (in 9.999514 seconds) chunkcleanupworker stdout | 2023-08-08 06:12:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:22 UTC)" (scheduled at 2023-08-08 06:12:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:12:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:12:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 12, 294752), True, datetime.datetime(2023, 8, 8, 6, 12, 12, 294752), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:12:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:12:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:12:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:22 UTC)" executed successfully builder stdout | 2023-08-08 06:12:12,771 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:12:12,780 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:12:12,780 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:12:12,785 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:12:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:12:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:54.284093+00:00 (in 41.186178 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:12:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:13 UTC)" (scheduled at 2023-08-08 06:12:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:12:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:12:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 13, 98314), True, datetime.datetime(2023, 8, 8, 6, 12, 13, 98314), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:12:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:12:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:12:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:12:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:12:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:25.371682+00:00 (in 9.999460 seconds) notificationworker stdout | 2023-08-08 06:12:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:25 UTC)" (scheduled at 2023-08-08 06:12:15.371682+00:00) notificationworker stdout | 2023-08-08 06:12:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:12:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 15, 372868), True, datetime.datetime(2023, 8, 8, 6, 12, 15, 372868), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:12:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:12:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:12:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:12:18,156 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:12:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:12:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:29.417660+00:00 (in 9.999404 seconds) gcworker stdout | 2023-08-08 06:12:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:29 UTC)" (scheduled at 2023-08-08 06:12:19.417660+00:00) gcworker stdout | 2023-08-08 06:12:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265539418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:12:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:12:19,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:12:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:29 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:12:19,448 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:12:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:12:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:26.890741+00:00 (in 7.202873 seconds) securityworker stdout | 2023-08-08 06:12:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:49 UTC)" (scheduled at 2023-08-08 06:12:19.687363+00:00) securityworker stdout | 2023-08-08 06:12:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:12:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:12:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:12:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:12:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:12:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:12:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 7, 19, 691895), 1, 2]) securityworker stdout | 2023-08-08 06:12:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 7, 19, 691895), 1, 2]) securityworker stdout | 2023-08-08 06:12:19,706 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:12:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:12:20,477 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:12:20,924 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:12:22,179 [200] [DEBUG] [app] Starting request: urn:request:e8962e86-9870-40b6-8c6e-b856164d8367 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:12:22,179 [198] [DEBUG] [app] Starting request: urn:request:4ea65cb1-09fb-47de-b4ce-c1fb991b5b64 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:12:22,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:22,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:22,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:12:22,184 [203] [DEBUG] [app] Starting request: urn:request:7655f573-ec75-4a9b-bf82-2a0966d42420 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:12:22,185 [203] [DEBUG] [app] Ending request: urn:request:7655f573-ec75-4a9b-bf82-2a0966d42420 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7655f573-ec75-4a9b-bf82-2a0966d42420', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:12:22,185 [210] [DEBUG] [app] Starting request: urn:request:8e9afb48-7294-4570-ae35-f8d1916ab935 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:12:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:12:22,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:12:22,186 [210] [DEBUG] [app] Ending request: urn:request:8e9afb48-7294-4570-ae35-f8d1916ab935 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8e9afb48-7294-4570-ae35-f8d1916ab935', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:12:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:12:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:22,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:22,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:22,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:22,190 [199] [DEBUG] [app] Starting request: urn:request:bacb0913-e747-461f-863b-13ff643e32a4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:22,191 [199] [DEBUG] [app] Ending request: urn:request:bacb0913-e747-461f-863b-13ff643e32a4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bacb0913-e747-461f-863b-13ff643e32a4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:12:22,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:12:22,191 [197] [DEBUG] [app] Starting request: urn:request:967bba49-fb9f-4408-a5f2-6e5ad852a5b4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:22,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:22,192 [197] [DEBUG] [app] Ending request: urn:request:967bba49-fb9f-4408-a5f2-6e5ad852a5b4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:967bba49-fb9f-4408-a5f2-6e5ad852a5b4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:12:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:12:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:12:22,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:22,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:12:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:12:22,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:12:22,203 [200] [DEBUG] [app] Ending request: urn:request:e8962e86-9870-40b6-8c6e-b856164d8367 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e8962e86-9870-40b6-8c6e-b856164d8367', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:12:22,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:12:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:12:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:12:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 06:12:22,204 [198] [DEBUG] [app] Ending request: urn:request:4ea65cb1-09fb-47de-b4ce-c1fb991b5b64 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4ea65cb1-09fb-47de-b4ce-c1fb991b5b64', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:12:22,204 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:12:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:12:22,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:12:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:12:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:12:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:28.874906+00:00 (in 6.580612 seconds) chunkcleanupworker stdout | 2023-08-08 06:12:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:32 UTC)" (scheduled at 2023-08-08 06:12:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:12:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:12:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 22, 294695), True, datetime.datetime(2023, 8, 8, 6, 12, 22, 294695), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:12:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:12:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:12:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:32 UTC)" executed successfully builder stdout | 2023-08-08 06:12:22,795 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:12:22,804 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:12:22,804 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:12:22,809 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:12:23,712 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:12:25,059 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:12:25,064 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:12:25,068 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:12:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:12:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:35.371682+00:00 (in 9.999475 seconds) notificationworker stdout | 2023-08-08 06:12:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:35 UTC)" (scheduled at 2023-08-08 06:12:25.371682+00:00) notificationworker stdout | 2023-08-08 06:12:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:12:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 25, 372775), True, datetime.datetime(2023, 8, 8, 6, 12, 25, 372775), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:12:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:12:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:12:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:35 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:12:25,471 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} exportactionlogsworker stdout | 2023-08-08 06:12:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:12:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:47.398551+00:00 (in 21.883638 seconds) exportactionlogsworker stdout | 2023-08-08 06:12:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:25 UTC)" (scheduled at 2023-08-08 06:12:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:12:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:12:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:25 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:12:25,713 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:12:25,942 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:12:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:12:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:49.687363+00:00 (in 22.796127 seconds) securityworker stdout | 2023-08-08 06:12:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:56 UTC)" (scheduled at 2023-08-08 06:12:26.890741+00:00) securityworker stdout | 2023-08-08 06:12:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:12:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:12:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:12:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:12:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:12:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:12:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:12:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:12:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:12:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:12:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 7, 26, 896375), 1, 2]) securityworker stdout | 2023-08-08 06:12:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 7, 26, 896375), 1, 2]) securityworker stdout | 2023-08-08 06:12:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:12:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:12:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:12:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:12:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:12:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:12:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:12:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:12:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:12:26,943 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:12:27,473 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} queuecleanupworker stdout | 2023-08-08 06:12:27,535 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:12:27,561 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} repositoryactioncounter stdout | 2023-08-08 06:12:27,581 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:12:27,606 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:12:27,634 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:12:27,639 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} chunkcleanupworker stdout | 2023-08-08 06:12:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:12:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:32.293757+00:00 (in 3.418339 seconds) chunkcleanupworker stdout | 2023-08-08 06:12:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:28 UTC)" (scheduled at 2023-08-08 06:12:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:12:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:12:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:28 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:12:28,897 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:12:29,108 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:12:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:12:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:39.417660+00:00 (in 9.999472 seconds) gcworker stdout | 2023-08-08 06:12:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:39 UTC)" (scheduled at 2023-08-08 06:12:29.417660+00:00) gcworker stdout | 2023-08-08 06:12:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265549418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:12:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:12:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:12:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:12:30,182 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:12:31,036 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:12:31,111 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:12:31,583 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:12:32,043 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:12:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:12:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:42.293757+00:00 (in 9.999494 seconds) chunkcleanupworker stdout | 2023-08-08 06:12:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:42 UTC)" (scheduled at 2023-08-08 06:12:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:12:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:12:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 32, 294746), True, datetime.datetime(2023, 8, 8, 6, 12, 32, 294746), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:12:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:12:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:12:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:42 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:12:32,433 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:12:32,700 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:12:32,819 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:12:32,829 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:12:32,830 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:12:32,834 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:12:33,346 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:12:33,349 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:12:33,353 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:12:33,358 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:12:33,363 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:12:33,369 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:12:33,373 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:12:33,376 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:12:33,379 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:12:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:12:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:45.371682+00:00 (in 9.999473 seconds) notificationworker stdout | 2023-08-08 06:12:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:45 UTC)" (scheduled at 2023-08-08 06:12:35.371682+00:00) notificationworker stdout | 2023-08-08 06:12:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:12:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 35, 372723), True, datetime.datetime(2023, 8, 8, 6, 12, 35, 372723), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:12:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:12:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:12:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:12:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:12:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:54.393627+00:00 (in 17.812940 seconds) namespacegcworker stdout | 2023-08-08 06:12:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:36 UTC)" (scheduled at 2023-08-08 06:12:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:12:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:12:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:12:37,179 [197] [DEBUG] [app] Starting request: urn:request:e8e8ed4c-0d87-42c4-9112-b93d6a2ec0db (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:12:37,179 [200] [DEBUG] [app] Starting request: urn:request:cf587eed-130e-412d-b430-4972dacd995c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:12:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:12:37,185 [203] [DEBUG] [app] Starting request: urn:request:3b2ad79b-2935-4527-8b33-bc35eb4b99fc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:12:37,185 [210] [DEBUG] [app] Starting request: urn:request:6b84aff6-6c05-4454-aa0d-6bff9020c775 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:12:37,185 [203] [DEBUG] [app] Ending request: urn:request:3b2ad79b-2935-4527-8b33-bc35eb4b99fc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3b2ad79b-2935-4527-8b33-bc35eb4b99fc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:12:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:12:37,186 [210] [DEBUG] [app] Ending request: urn:request:6b84aff6-6c05-4454-aa0d-6bff9020c775 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6b84aff6-6c05-4454-aa0d-6bff9020c775', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:12:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:12:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:12:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:37,191 [197] [DEBUG] [app] Starting request: urn:request:437566df-f1c5-4a89-a804-8e3dd4cbf363 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:37,191 [198] [DEBUG] [app] Starting request: urn:request:567a9d1f-730a-45ef-a449-eefc1f28fe37 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:37,191 [197] [DEBUG] [app] Ending request: urn:request:437566df-f1c5-4a89-a804-8e3dd4cbf363 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:437566df-f1c5-4a89-a804-8e3dd4cbf363', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:12:37,191 [198] [DEBUG] [app] Ending request: urn:request:567a9d1f-730a-45ef-a449-eefc1f28fe37 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:567a9d1f-730a-45ef-a449-eefc1f28fe37', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:12:37,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:12:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:12:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:12:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:12:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:12:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:12:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:12:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:07.197412+00:00 (in 29.999527 seconds) buildlogsarchiver stdout | 2023-08-08 06:12:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:07 UTC)" (scheduled at 2023-08-08 06:12:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:12:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 12, 37, 198359), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:12:37,202 [197] [DEBUG] [app] Ending request: urn:request:e8e8ed4c-0d87-42c4-9112-b93d6a2ec0db (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e8e8ed4c-0d87-42c4-9112-b93d6a2ec0db', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:12:37,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:12:37,203 [200] [DEBUG] [app] Ending request: urn:request:cf587eed-130e-412d-b430-4972dacd995c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cf587eed-130e-412d-b430-4972dacd995c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:12:37,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:12:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:12:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:12:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:12:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:12:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:12:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:12:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:12:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:12:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:12:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:12:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:12:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:12:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:49.417660+00:00 (in 9.999482 seconds) gcworker stdout | 2023-08-08 06:12:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:49 UTC)" (scheduled at 2023-08-08 06:12:39.417660+00:00) gcworker stdout | 2023-08-08 06:12:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265559418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:12:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:12:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:12:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:12:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:12:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:52.293757+00:00 (in 9.999493 seconds) chunkcleanupworker stdout | 2023-08-08 06:12:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:52 UTC)" (scheduled at 2023-08-08 06:12:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:12:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:12:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 42, 294730), True, datetime.datetime(2023, 8, 8, 6, 12, 42, 294730), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:12:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:12:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:12:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:52 UTC)" executed successfully builder stdout | 2023-08-08 06:12:42,844 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:12:42,853 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:12:42,853 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:12:42,857 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:12:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:12:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:55.371682+00:00 (in 9.999480 seconds) notificationworker stdout | 2023-08-08 06:12:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:55 UTC)" (scheduled at 2023-08-08 06:12:45.371682+00:00) notificationworker stdout | 2023-08-08 06:12:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:12:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 45, 372740), True, datetime.datetime(2023, 8, 8, 6, 12, 45, 372740), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:12:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:12:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:12:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:12:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:12:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:25.514341+00:00 (in 38.115285 seconds) exportactionlogsworker stdout | 2023-08-08 06:12:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:47 UTC)" (scheduled at 2023-08-08 06:12:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:12:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:12:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 47, 399564), True, datetime.datetime(2023, 8, 8, 6, 12, 47, 399564), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:12:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:12:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:12:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:12:48,185 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:12:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:12:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:59.417660+00:00 (in 9.999489 seconds) gcworker stdout | 2023-08-08 06:12:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:59 UTC)" (scheduled at 2023-08-08 06:12:49.417660+00:00) gcworker stdout | 2023-08-08 06:12:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265569418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:12:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:12:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:12:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:12:59 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:12:49,456 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:12:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:12:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:12:56.890741+00:00 (in 7.202894 seconds) securityworker stdout | 2023-08-08 06:12:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:19 UTC)" (scheduled at 2023-08-08 06:12:49.687363+00:00) securityworker stdout | 2023-08-08 06:12:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:12:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:12:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:12:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:12:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:12:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:12:50,490 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:12:50,962 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:12:52,180 [200] [DEBUG] [app] Starting request: urn:request:fe6bdfdc-ee8a-4a30-a13a-9f4f2a1d9884 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:12:52,180 [197] [DEBUG] [app] Starting request: urn:request:c8f0d60a-272d-478f-8dd9-aa68966e7fcb (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:12:52,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:52,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:52,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:12:52,185 [210] [DEBUG] [app] Starting request: urn:request:292361b0-1ae4-4a44-99cb-063f61e35254 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:12:52,186 [203] [DEBUG] [app] Starting request: urn:request:e6d21fd1-6047-4c2a-9804-334fda43d34a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:12:52,186 [210] [DEBUG] [app] Ending request: urn:request:292361b0-1ae4-4a44-99cb-063f61e35254 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:292361b0-1ae4-4a44-99cb-063f61e35254', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:12:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:12:52,186 [203] [DEBUG] [app] Ending request: urn:request:e6d21fd1-6047-4c2a-9804-334fda43d34a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e6d21fd1-6047-4c2a-9804-334fda43d34a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:12:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:12:52,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:12:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:52,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:12:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:52,192 [197] [DEBUG] [app] Starting request: urn:request:df51b423-0b9f-45fc-b25a-06da087ed83d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:52,192 [197] [DEBUG] [app] Ending request: urn:request:df51b423-0b9f-45fc-b25a-06da087ed83d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:df51b423-0b9f-45fc-b25a-06da087ed83d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:12:52,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:12:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:12:52,194 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:12:52,195 [198] [DEBUG] [app] Starting request: urn:request:9ef799b0-bc3a-4ea2-aa6f-bc1bc553908d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:12:52,195 [198] [DEBUG] [app] Ending request: urn:request:9ef799b0-bc3a-4ea2-aa6f-bc1bc553908d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9ef799b0-bc3a-4ea2-aa6f-bc1bc553908d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:12:52,196 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:12:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:12:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:12:52,196 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:12:52,196 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:12:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:12:52,200 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:12:52,203 [200] [DEBUG] [app] Ending request: urn:request:fe6bdfdc-ee8a-4a30-a13a-9f4f2a1d9884 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fe6bdfdc-ee8a-4a30-a13a-9f4f2a1d9884', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:12:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:12:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:12:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:12:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:12:52,205 [197] [DEBUG] [app] Ending request: urn:request:c8f0d60a-272d-478f-8dd9-aa68966e7fcb (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c8f0d60a-272d-478f-8dd9-aa68966e7fcb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:12:52,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:12:52,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:12:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:12:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:12:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:12:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:02.293757+00:00 (in 9.999511 seconds) chunkcleanupworker stdout | 2023-08-08 06:12:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:02 UTC)" (scheduled at 2023-08-08 06:12:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:12:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:12:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 52, 294750), True, datetime.datetime(2023, 8, 8, 6, 12, 52, 294750), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:12:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:12:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:12:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:02 UTC)" executed successfully builder stdout | 2023-08-08 06:12:52,864 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:12:52,873 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:12:52,873 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:12:52,878 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:12:53,735 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:12:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:12:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:13.097375+00:00 (in 18.812743 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:12:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:54 UTC)" (scheduled at 2023-08-08 06:12:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:12:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:12:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:12:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:12:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:36.579841+00:00 (in 42.185690 seconds) namespacegcworker stdout | 2023-08-08 06:12:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:54 UTC)" (scheduled at 2023-08-08 06:12:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:12:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:12:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 54, 394735), True, datetime.datetime(2023, 8, 8, 6, 12, 54, 394735), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:12:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:12:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:12:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:13:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:12:55,067 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:12:55,073 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:12:55,076 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:12:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:12:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:02.178530+00:00 (in 6.806318 seconds) notificationworker stdout | 2023-08-08 06:12:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:05 UTC)" (scheduled at 2023-08-08 06:12:55.371682+00:00) notificationworker stdout | 2023-08-08 06:12:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:12:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 12, 55, 372786), True, datetime.datetime(2023, 8, 8, 6, 12, 55, 372786), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:12:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:12:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:12:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:12:55,478 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:12:55,741 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:12:55,980 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:12:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:12:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:19.687363+00:00 (in 22.796129 seconds) securityworker stdout | 2023-08-08 06:12:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:26 UTC)" (scheduled at 2023-08-08 06:12:56.890741+00:00) securityworker stdout | 2023-08-08 06:12:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:12:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:12:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:12:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:12:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:12:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:12:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:12:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:12:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:12:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:12:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 7, 56, 897375), 1, 2]) securityworker stdout | 2023-08-08 06:12:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 7, 56, 897375), 1, 2]) securityworker stdout | 2023-08-08 06:12:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:12:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:12:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:12:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:12:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:12:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:12:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:12:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:12:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:12:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:12:56,962 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:12:57,482 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} queuecleanupworker stdout | 2023-08-08 06:12:57,569 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:12:57,572 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} repositoryactioncounter stdout | 2023-08-08 06:12:57,614 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:12:57,618 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:12:57,645 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:12:57,649 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:12:58,931 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:12:59,138 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:12:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:12:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:09.417660+00:00 (in 9.999471 seconds) gcworker stdout | 2023-08-08 06:12:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:09 UTC)" (scheduled at 2023-08-08 06:12:59.417660+00:00) gcworker stdout | 2023-08-08 06:12:59,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265579418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:12:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:12:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:12:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:13:00,202 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:13:01,072 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:13:01,137 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:13:01,620 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:13:02,068 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} notificationworker stdout | 2023-08-08 06:13:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:13:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:05.371682+00:00 (in 3.192588 seconds) notificationworker stdout | 2023-08-08 06:13:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:02 UTC)" (scheduled at 2023-08-08 06:13:02.178530+00:00) notificationworker stdout | 2023-08-08 06:13:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:13:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:13:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:13:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:12.293757+00:00 (in 9.999513 seconds) chunkcleanupworker stdout | 2023-08-08 06:13:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:12 UTC)" (scheduled at 2023-08-08 06:13:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:13:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:13:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 2, 294742), True, datetime.datetime(2023, 8, 8, 6, 13, 2, 294742), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:13:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:13:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:13:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:12 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:13:02,458 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:13:02,726 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:13:02,888 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:13:02,897 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:13:02,897 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:13:02,903 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:13:03,355 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:13:03,358 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:13:03,360 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:13:03,365 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:13:03,372 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:13:03,379 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:13:03,384 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:13:03,387 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:13:03,389 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:13:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:13:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:09.793658+00:00 (in 5.605683 seconds) repositorygcworker stdout | 2023-08-08 06:13:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:04 UTC)" (scheduled at 2023-08-08 06:13:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:13:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:13:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:13:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:13:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:15.371682+00:00 (in 9.999478 seconds) notificationworker stdout | 2023-08-08 06:13:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:15 UTC)" (scheduled at 2023-08-08 06:13:05.371682+00:00) notificationworker stdout | 2023-08-08 06:13:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:13:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 5, 372708), True, datetime.datetime(2023, 8, 8, 6, 13, 5, 372708), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:13:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:13:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:13:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:13:07,178 [200] [DEBUG] [app] Starting request: urn:request:11d5abbf-55eb-4be6-85f5-4ad6a67475c7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:13:07,179 [197] [DEBUG] [app] Starting request: urn:request:7693dace-744b-4e79-a577-362e7bcf83df (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:13:07,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:13:07,184 [203] [DEBUG] [app] Starting request: urn:request:f8c865cb-d099-40b4-b618-971dd65730f0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:13:07,185 [209] [DEBUG] [app] Starting request: urn:request:ea77855e-26a0-4a2e-9ee5-5288b5ce2d29 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:13:07,185 [203] [DEBUG] [app] Ending request: urn:request:f8c865cb-d099-40b4-b618-971dd65730f0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f8c865cb-d099-40b4-b618-971dd65730f0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:13:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:13:07,185 [209] [DEBUG] [app] Ending request: urn:request:ea77855e-26a0-4a2e-9ee5-5288b5ce2d29 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ea77855e-26a0-4a2e-9ee5-5288b5ce2d29', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:13:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:07,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:13:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:07,191 [197] [DEBUG] [app] Starting request: urn:request:dee84fc9-5f6d-431c-9fa7-6bd94621868e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:13:07,191 [197] [DEBUG] [app] Ending request: urn:request:dee84fc9-5f6d-431c-9fa7-6bd94621868e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:dee84fc9-5f6d-431c-9fa7-6bd94621868e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:13:07,192 [198] [DEBUG] [app] Starting request: urn:request:08808189-49de-42e7-a1cf-9e45d9003660 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:13:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:13:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:07,192 [198] [DEBUG] [app] Ending request: urn:request:08808189-49de-42e7-a1cf-9e45d9003660 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:08808189-49de-42e7-a1cf-9e45d9003660', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:13:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:13:07,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:13:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:13:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:13:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:37.197412+00:00 (in 29.999480 seconds) gunicorn-web stdout | 2023-08-08 06:13:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:13:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:37 UTC)" (scheduled at 2023-08-08 06:13:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:13:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 13, 7, 198343), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:13:07,202 [197] [DEBUG] [app] Ending request: urn:request:7693dace-744b-4e79-a577-362e7bcf83df (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7693dace-744b-4e79-a577-362e7bcf83df', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:13:07,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:13:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:13:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:13:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:13:07,203 [200] [DEBUG] [app] Ending request: urn:request:11d5abbf-55eb-4be6-85f5-4ad6a67475c7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:11d5abbf-55eb-4be6-85f5-4ad6a67475c7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:13:07,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:13:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:13:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:13:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:13:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:13:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:13:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:13:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:13:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:13:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:13:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:19.417660+00:00 (in 9.999488 seconds) gcworker stdout | 2023-08-08 06:13:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:19 UTC)" (scheduled at 2023-08-08 06:13:09.417660+00:00) gcworker stdout | 2023-08-08 06:13:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265589419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:13:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:13:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:13:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:13:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:13:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:04.187429+00:00 (in 54.393250 seconds) repositorygcworker stdout | 2023-08-08 06:13:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:09 UTC)" (scheduled at 2023-08-08 06:13:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:13:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:13:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 9, 795147), True, datetime.datetime(2023, 8, 8, 6, 13, 9, 795147), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:13:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:13:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:13:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:13:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:13:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:22.293757+00:00 (in 9.999501 seconds) chunkcleanupworker stdout | 2023-08-08 06:13:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:22 UTC)" (scheduled at 2023-08-08 06:13:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:13:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:13:12,298 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 12, 294724), True, datetime.datetime(2023, 8, 8, 6, 13, 12, 294724), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:13:12,306 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:13:12,306 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:13:12,306 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:22 UTC)" executed successfully builder stdout | 2023-08-08 06:13:12,904 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:13:12,914 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:13:12,914 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:13:12,920 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:13:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:13:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:54.284093+00:00 (in 41.186134 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:13:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:13 UTC)" (scheduled at 2023-08-08 06:13:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:13:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:13:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 13, 98504), True, datetime.datetime(2023, 8, 8, 6, 13, 13, 98504), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:13:13,105 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:13:13,105 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:13:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:13:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:13:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:25.371682+00:00 (in 9.999471 seconds) notificationworker stdout | 2023-08-08 06:13:15,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:25 UTC)" (scheduled at 2023-08-08 06:13:15.371682+00:00) notificationworker stdout | 2023-08-08 06:13:15,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:13:15,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 15, 373233), True, datetime.datetime(2023, 8, 8, 6, 13, 15, 373233), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:13:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:13:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:13:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:13:18,193 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:13:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:13:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:29.417660+00:00 (in 9.999473 seconds) gcworker stdout | 2023-08-08 06:13:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:29 UTC)" (scheduled at 2023-08-08 06:13:19.417660+00:00) gcworker stdout | 2023-08-08 06:13:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265599418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:13:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:13:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:13:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:29 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:13:19,492 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:13:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:13:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:26.890741+00:00 (in 7.202904 seconds) securityworker stdout | 2023-08-08 06:13:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:49 UTC)" (scheduled at 2023-08-08 06:13:19.687363+00:00) securityworker stdout | 2023-08-08 06:13:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:13:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:13:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:13:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:13:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:13:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:13:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 8, 19, 691888), 1, 2]) securityworker stdout | 2023-08-08 06:13:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 8, 19, 691888), 1, 2]) securityworker stdout | 2023-08-08 06:13:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:13:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:13:20,513 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:13:20,999 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:13:22,178 [197] [DEBUG] [app] Starting request: urn:request:fd9736c7-4bbe-4b1b-8c64-04ff628634d0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:13:22,178 [199] [DEBUG] [app] Starting request: urn:request:5f439f99-c9d2-4978-99fc-11de23337557 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:13:22,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:22,182 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:13:22,184 [209] [DEBUG] [app] Starting request: urn:request:6e929724-68a6-4baa-a649-cbfe75802028 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:13:22,184 [203] [DEBUG] [app] Starting request: urn:request:b4e14c94-a075-43b0-8cd0-420bde33ec95 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:13:22,185 [209] [DEBUG] [app] Ending request: urn:request:6e929724-68a6-4baa-a649-cbfe75802028 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6e929724-68a6-4baa-a649-cbfe75802028', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:13:22,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:13:22,185 [203] [DEBUG] [app] Ending request: urn:request:b4e14c94-a075-43b0-8cd0-420bde33ec95 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b4e14c94-a075-43b0-8cd0-420bde33ec95', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:13:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:13:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:22,190 [199] [DEBUG] [app] Starting request: urn:request:edc76def-f17a-45cf-8d18-253baf320113 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:13:22,191 [199] [DEBUG] [app] Ending request: urn:request:edc76def-f17a-45cf-8d18-253baf320113 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:edc76def-f17a-45cf-8d18-253baf320113', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:13:22,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:22,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:13:22,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:22,193 [198] [DEBUG] [app] Starting request: urn:request:3eab8bef-4440-40f9-82a0-23d5822eeb32 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:13:22,193 [198] [DEBUG] [app] Ending request: urn:request:3eab8bef-4440-40f9-82a0-23d5822eeb32 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3eab8bef-4440-40f9-82a0-23d5822eeb32', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:13:22,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:22,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:22,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:13:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:13:22,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:13:22,202 [197] [DEBUG] [app] Ending request: urn:request:fd9736c7-4bbe-4b1b-8c64-04ff628634d0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fd9736c7-4bbe-4b1b-8c64-04ff628634d0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:13:22,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:13:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:13:22,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:13:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:13:22,204 [199] [DEBUG] [app] Ending request: urn:request:5f439f99-c9d2-4978-99fc-11de23337557 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5f439f99-c9d2-4978-99fc-11de23337557', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:13:22,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:13:22,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:13:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:13:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) chunkcleanupworker stdout | 2023-08-08 06:13:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:13:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:28.874906+00:00 (in 6.580671 seconds) chunkcleanupworker stdout | 2023-08-08 06:13:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:32 UTC)" (scheduled at 2023-08-08 06:13:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:13:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:13:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 22, 294707), True, datetime.datetime(2023, 8, 8, 6, 13, 22, 294707), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:13:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:13:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:13:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:32 UTC)" executed successfully builder stdout | 2023-08-08 06:13:22,928 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:13:22,938 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:13:22,938 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:13:22,943 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:13:23,754 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:13:25,076 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:13:25,080 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:13:25,083 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:13:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:13:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:35.371682+00:00 (in 9.999411 seconds) notificationworker stdout | 2023-08-08 06:13:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:35 UTC)" (scheduled at 2023-08-08 06:13:25.371682+00:00) notificationworker stdout | 2023-08-08 06:13:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:13:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 25, 372671), True, datetime.datetime(2023, 8, 8, 6, 13, 25, 372671), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:13:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:13:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:13:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:35 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:13:25,491 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} exportactionlogsworker stdout | 2023-08-08 06:13:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:13:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:33.514094+00:00 (in 7.999232 seconds) exportactionlogsworker stdout | 2023-08-08 06:13:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:25 UTC)" (scheduled at 2023-08-08 06:13:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:13:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:13:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:25 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:13:25,775 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:13:26,018 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:13:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:13:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:49.687363+00:00 (in 22.796068 seconds) securityworker stdout | 2023-08-08 06:13:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:56 UTC)" (scheduled at 2023-08-08 06:13:26.890741+00:00) securityworker stdout | 2023-08-08 06:13:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:13:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:13:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:13:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:13:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:13:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:13:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:13:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:13:26,971 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:13:26,978 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:13:26,980 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:26,980 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:26,981 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 8, 26, 971126), 1, 2]) securityworker stdout | 2023-08-08 06:13:26,982 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:26,982 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:26,982 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 8, 26, 971126), 1, 2]) securityworker stdout | 2023-08-08 06:13:26,984 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:26,984 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:26,984 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:13:26,984 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:13:26,984 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:13:26,984 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:13:26,984 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:13:26,985 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:13:26,985 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:13:26,985 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:13:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:13:26,998 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:13:27,491 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:13:27,584 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} queuecleanupworker stdout | 2023-08-08 06:13:27,606 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:13:27,626 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:13:27,632 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:13:27,662 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:13:27,666 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} chunkcleanupworker stdout | 2023-08-08 06:13:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:13:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:32.293757+00:00 (in 3.418303 seconds) chunkcleanupworker stdout | 2023-08-08 06:13:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:28 UTC)" (scheduled at 2023-08-08 06:13:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:13:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:13:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:28 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:13:28,969 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:13:29,146 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:13:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:13:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:39.417660+00:00 (in 9.999382 seconds) gcworker stdout | 2023-08-08 06:13:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:39 UTC)" (scheduled at 2023-08-08 06:13:29.417660+00:00) gcworker stdout | 2023-08-08 06:13:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265609418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:13:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:13:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:13:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:13:30,210 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:13:31,108 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:13:31,174 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:13:31,657 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:13:32,095 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:13:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:13:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:42.293757+00:00 (in 9.999436 seconds) chunkcleanupworker stdout | 2023-08-08 06:13:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:42 UTC)" (scheduled at 2023-08-08 06:13:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:13:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:13:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 32, 294892), True, datetime.datetime(2023, 8, 8, 6, 13, 32, 294892), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:13:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:13:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:13:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:42 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:13:32,491 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:13:32,764 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:13:32,953 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:13:32,962 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:13:32,962 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:13:32,967 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:13:33,365 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:13:33,368 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:13:33,373 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:13:33,376 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:13:33,381 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:13:33,388 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:13:33,392 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:13:33,396 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:13:33,399 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} exportactionlogsworker stdout | 2023-08-08 06:13:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:13:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:47.398551+00:00 (in 13.883914 seconds) exportactionlogsworker stdout | 2023-08-08 06:13:33,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:18:33 UTC)" (scheduled at 2023-08-08 06:13:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:13:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 13, 33, 515181), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:13:33,523 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 13, 33, 515181), True, datetime.datetime(2023, 8, 8, 6, 13, 33, 515181), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:13:33,525 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 13, 33, 515181), True, datetime.datetime(2023, 8, 8, 6, 13, 33, 515181), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 13, 33, 515181), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:13:33,526 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:13:33,526 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:18:33 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:13:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:13:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:45.371682+00:00 (in 9.999489 seconds) notificationworker stdout | 2023-08-08 06:13:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:45 UTC)" (scheduled at 2023-08-08 06:13:35.371682+00:00) notificationworker stdout | 2023-08-08 06:13:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:13:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 35, 372565), True, datetime.datetime(2023, 8, 8, 6, 13, 35, 372565), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:13:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:13:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:13:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:13:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:13:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:54.393627+00:00 (in 17.813278 seconds) namespacegcworker stdout | 2023-08-08 06:13:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:36 UTC)" (scheduled at 2023-08-08 06:13:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:13:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:13:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:13:37,179 [200] [DEBUG] [app] Starting request: urn:request:05e04853-0e34-45ef-8052-3132f2bf7a7b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:13:37,179 [197] [DEBUG] [app] Starting request: urn:request:a5227750-5d25-48c6-993b-b624d8f7fb3f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:13:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:13:37,184 [207] [DEBUG] [app] Starting request: urn:request:a31bd6f6-f42e-40c4-98c9-2fa1a0d240b4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:13:37,185 [207] [DEBUG] [app] Ending request: urn:request:a31bd6f6-f42e-40c4-98c9-2fa1a0d240b4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a31bd6f6-f42e-40c4-98c9-2fa1a0d240b4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:13:37,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:13:37,185 [206] [DEBUG] [app] Starting request: urn:request:1fab6aab-9b98-48f5-b3f6-3fd058f7076a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:13:37,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:13:37,186 [206] [DEBUG] [app] Ending request: urn:request:1fab6aab-9b98-48f5-b3f6-3fd058f7076a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1fab6aab-9b98-48f5-b3f6-3fd058f7076a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:13:37,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:37,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:37,191 [200] [DEBUG] [app] Starting request: urn:request:cd62ae20-8671-40bb-a7f6-60bfa811021c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:13:37,191 [200] [DEBUG] [app] Ending request: urn:request:cd62ae20-8671-40bb-a7f6-60bfa811021c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:cd62ae20-8671-40bb-a7f6-60bfa811021c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:13:37,191 [198] [DEBUG] [app] Starting request: urn:request:e2129891-9c32-4fc9-ad4e-ec1c931b98b6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:13:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:37,192 [198] [DEBUG] [app] Ending request: urn:request:e2129891-9c32-4fc9-ad4e-ec1c931b98b6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e2129891-9c32-4fc9-ad4e-ec1c931b98b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:13:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:13:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:13:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:13:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:13:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:13:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:13:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:07.197412+00:00 (in 29.999486 seconds) buildlogsarchiver stdout | 2023-08-08 06:13:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:07 UTC)" (scheduled at 2023-08-08 06:13:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:13:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 13, 37, 198984), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:13:37,202 [200] [DEBUG] [app] Ending request: urn:request:05e04853-0e34-45ef-8052-3132f2bf7a7b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:05e04853-0e34-45ef-8052-3132f2bf7a7b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:13:37,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:13:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:13:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:13:37,203 [197] [DEBUG] [app] Ending request: urn:request:a5227750-5d25-48c6-993b-b624d8f7fb3f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a5227750-5d25-48c6-993b-b624d8f7fb3f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:13:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:13:37,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:13:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:13:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:13:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:13:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:13:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:13:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:13:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:13:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:13:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:13:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:49.417660+00:00 (in 9.999421 seconds) gcworker stdout | 2023-08-08 06:13:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:49 UTC)" (scheduled at 2023-08-08 06:13:39.417660+00:00) gcworker stdout | 2023-08-08 06:13:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265619418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:13:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:13:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:13:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:13:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:13:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:52.293757+00:00 (in 9.999505 seconds) chunkcleanupworker stdout | 2023-08-08 06:13:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:52 UTC)" (scheduled at 2023-08-08 06:13:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:13:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:13:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 42, 294715), True, datetime.datetime(2023, 8, 8, 6, 13, 42, 294715), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:13:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:13:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:13:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:52 UTC)" executed successfully builder stdout | 2023-08-08 06:13:42,977 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:13:42,986 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:13:42,986 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:13:42,990 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:13:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:13:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:49.178248+00:00 (in 3.806010 seconds) notificationworker stdout | 2023-08-08 06:13:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:55 UTC)" (scheduled at 2023-08-08 06:13:45.371682+00:00) notificationworker stdout | 2023-08-08 06:13:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:13:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 45, 372643), True, datetime.datetime(2023, 8, 8, 6, 13, 45, 372643), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:13:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:13:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:13:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:13:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:13:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:25.514341+00:00 (in 38.115263 seconds) exportactionlogsworker stdout | 2023-08-08 06:13:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:47 UTC)" (scheduled at 2023-08-08 06:13:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:13:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:13:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 47, 399533), True, datetime.datetime(2023, 8, 8, 6, 13, 47, 399533), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:13:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:13:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:13:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:13:48,231 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} notificationworker stdout | 2023-08-08 06:13:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:13:49,178 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:55.371682+00:00 (in 6.192883 seconds) notificationworker stdout | 2023-08-08 06:13:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:18:49 UTC)" (scheduled at 2023-08-08 06:13:49.178248+00:00) notificationworker stdout | 2023-08-08 06:13:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 13, 49, 179124), 'notification/%']) notificationworker stdout | 2023-08-08 06:13:49,186 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 13, 49, 179124), True, datetime.datetime(2023, 8, 8, 6, 13, 49, 179124), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:13:49,188 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 13, 49, 179124), True, datetime.datetime(2023, 8, 8, 6, 13, 49, 179124), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 13, 49, 179124), 'notification/%']) notificationworker stdout | 2023-08-08 06:13:49,189 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:13:49,189 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:18:49 UTC)" executed successfully gcworker stdout | 2023-08-08 06:13:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:13:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:59.417660+00:00 (in 9.999386 seconds) gcworker stdout | 2023-08-08 06:13:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:59 UTC)" (scheduled at 2023-08-08 06:13:49.417660+00:00) gcworker stdout | 2023-08-08 06:13:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265629418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:13:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:13:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:13:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:13:59 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:13:49,527 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:13:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:13:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:13:56.890741+00:00 (in 7.202826 seconds) securityworker stdout | 2023-08-08 06:13:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:19 UTC)" (scheduled at 2023-08-08 06:13:49.687363+00:00) securityworker stdout | 2023-08-08 06:13:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:13:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:13:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:13:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:13:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:13:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:13:50,546 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:13:51,018 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:13:52,179 [197] [DEBUG] [app] Starting request: urn:request:dd8c0e3d-d6f3-4d04-89d6-48a75629d061 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:13:52,179 [200] [DEBUG] [app] Starting request: urn:request:69867c90-10b7-429b-9f8f-a193fee928c7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:13:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:52,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:13:52,186 [208] [DEBUG] [app] Starting request: urn:request:ef7d41f6-448e-4669-9785-75633a64e90a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:13:52,186 [210] [DEBUG] [app] Starting request: urn:request:ea14b813-5480-40ec-9b58-0300a6c4731d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:13:52,186 [208] [DEBUG] [app] Ending request: urn:request:ef7d41f6-448e-4669-9785-75633a64e90a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ef7d41f6-448e-4669-9785-75633a64e90a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:13:52,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:13:52,187 [210] [DEBUG] [app] Ending request: urn:request:ea14b813-5480-40ec-9b58-0300a6c4731d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ea14b813-5480-40ec-9b58-0300a6c4731d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:13:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:13:52,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:52,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:52,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:13:52,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:52,193 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:13:52,194 [199] [DEBUG] [app] Starting request: urn:request:345b646d-1116-4dc5-a739-44b6c03ed55c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:13:52,194 [198] [DEBUG] [app] Starting request: urn:request:fa5b2e4c-e28f-48da-bd18-1b174141b47b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:13:52,195 [199] [DEBUG] [app] Ending request: urn:request:345b646d-1116-4dc5-a739-44b6c03ed55c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:345b646d-1116-4dc5-a739-44b6c03ed55c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:13:52,195 [198] [DEBUG] [app] Ending request: urn:request:fa5b2e4c-e28f-48da-bd18-1b174141b47b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fa5b2e4c-e28f-48da-bd18-1b174141b47b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:13:52,195 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:13:52,195 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:52,195 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:13:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:13:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:13:52,195 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:13:52,196 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:13:52,196 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:13:52,201 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:13:52,201 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:13:52,206 [197] [DEBUG] [app] Ending request: urn:request:dd8c0e3d-d6f3-4d04-89d6-48a75629d061 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:dd8c0e3d-d6f3-4d04-89d6-48a75629d061', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:13:52,206 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:13:52,206 [200] [DEBUG] [app] Ending request: urn:request:69867c90-10b7-429b-9f8f-a193fee928c7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:69867c90-10b7-429b-9f8f-a193fee928c7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:13:52,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:13:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:13:52,206 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:13:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.028) gunicorn-web stdout | 2023-08-08 06:13:52,207 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:13:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:13:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.029) chunkcleanupworker stdout | 2023-08-08 06:13:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:13:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:02.293757+00:00 (in 9.999476 seconds) chunkcleanupworker stdout | 2023-08-08 06:13:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:02 UTC)" (scheduled at 2023-08-08 06:13:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:13:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:13:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 52, 294857), True, datetime.datetime(2023, 8, 8, 6, 13, 52, 294857), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:13:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:13:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:13:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:02 UTC)" executed successfully builder stdout | 2023-08-08 06:13:53,000 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:13:53,009 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:13:53,009 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:13:53,014 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:13:53,771 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:13:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:13:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:13.097375+00:00 (in 18.812613 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:13:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:54 UTC)" (scheduled at 2023-08-08 06:13:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:13:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:13:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:13:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:13:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:36.579841+00:00 (in 42.185555 seconds) namespacegcworker stdout | 2023-08-08 06:13:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:54 UTC)" (scheduled at 2023-08-08 06:13:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:13:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:13:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 54, 394852), True, datetime.datetime(2023, 8, 8, 6, 13, 54, 394852), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:13:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:13:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:13:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:14:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:13:55,085 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:13:55,088 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:13:55,090 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:13:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:13:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:02.178530+00:00 (in 6.806289 seconds) notificationworker stdout | 2023-08-08 06:13:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:05 UTC)" (scheduled at 2023-08-08 06:13:55.371682+00:00) notificationworker stdout | 2023-08-08 06:13:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:13:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 13, 55, 372602), True, datetime.datetime(2023, 8, 8, 6, 13, 55, 372602), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:13:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:13:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:13:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:13:55,522 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:13:55,796 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:13:56,047 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:13:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:13:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:19.687363+00:00 (in 22.796091 seconds) securityworker stdout | 2023-08-08 06:13:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:26 UTC)" (scheduled at 2023-08-08 06:13:56.890741+00:00) securityworker stdout | 2023-08-08 06:13:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:13:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:13:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:13:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:13:56,894 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:13:56,894 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:13:56,894 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:13:56,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:13:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:13:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:13:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 8, 56, 897562), 1, 2]) securityworker stdout | 2023-08-08 06:13:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 8, 56, 897562), 1, 2]) securityworker stdout | 2023-08-08 06:13:56,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:13:56,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:13:56,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:13:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:13:56,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:13:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:13:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:13:56,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:13:56,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:13:56,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:13:57,006 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:13:57,500 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:13:57,597 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} queuecleanupworker stdout | 2023-08-08 06:13:57,635 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:13:57,645 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} repositoryactioncounter stdout | 2023-08-08 06:13:57,662 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:13:57,674 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:13:57,677 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} buildlogsarchiver stdout | 2023-08-08 06:13:59,008 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:13:59,157 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:13:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:13:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:09.417660+00:00 (in 9.999443 seconds) gcworker stdout | 2023-08-08 06:13:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:09 UTC)" (scheduled at 2023-08-08 06:13:59.417660+00:00) gcworker stdout | 2023-08-08 06:13:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265639418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:13:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:13:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:13:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:14:00,226 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:14:01,123 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:14:01,212 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:14:01,681 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:14:02,132 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} notificationworker stdout | 2023-08-08 06:14:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:14:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:05.371682+00:00 (in 3.192660 seconds) notificationworker stdout | 2023-08-08 06:14:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:02 UTC)" (scheduled at 2023-08-08 06:14:02.178530+00:00) notificationworker stdout | 2023-08-08 06:14:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:14:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:14:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:14:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:12.293757+00:00 (in 9.999488 seconds) chunkcleanupworker stdout | 2023-08-08 06:14:02,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:12 UTC)" (scheduled at 2023-08-08 06:14:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:14:02,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:14:02,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 2, 295222), True, datetime.datetime(2023, 8, 8, 6, 14, 2, 295222), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:14:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:14:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:14:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:12 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:14:02,500 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:14:02,793 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:14:03,024 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:14:03,033 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:14:03,033 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:14:03,037 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:14:03,375 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:14:03,378 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:14:03,381 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:14:03,385 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:14:03,390 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:14:03,397 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:14:03,403 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:14:03,406 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:14:03,409 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:14:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:14:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:09.793658+00:00 (in 5.605721 seconds) repositorygcworker stdout | 2023-08-08 06:14:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:04 UTC)" (scheduled at 2023-08-08 06:14:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:14:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:14:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:14:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:14:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:15.371682+00:00 (in 9.999453 seconds) notificationworker stdout | 2023-08-08 06:14:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:15 UTC)" (scheduled at 2023-08-08 06:14:05.371682+00:00) notificationworker stdout | 2023-08-08 06:14:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:14:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 5, 372572), True, datetime.datetime(2023, 8, 8, 6, 14, 5, 372572), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:14:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:14:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:14:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:14:07,179 [199] [DEBUG] [app] Starting request: urn:request:661eddef-7f30-457b-baa5-49e628c02357 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:14:07,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:07,181 [197] [DEBUG] [app] Starting request: urn:request:e44d8e14-15b5-4e0b-ae4f-7d55e5669b56 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:14:07,183 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:14:07,184 [203] [DEBUG] [app] Starting request: urn:request:a7943cf9-9594-475c-93c7-de9fe13f8fe6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:14:07,185 [203] [DEBUG] [app] Ending request: urn:request:a7943cf9-9594-475c-93c7-de9fe13f8fe6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a7943cf9-9594-475c-93c7-de9fe13f8fe6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:14:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:14:07,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:07,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:14:07,186 [203] [DEBUG] [app] Starting request: urn:request:cebdbd7e-bdbe-4382-846c-1f98e2b3f635 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:14:07,187 [203] [DEBUG] [app] Ending request: urn:request:cebdbd7e-bdbe-4382-846c-1f98e2b3f635 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:cebdbd7e-bdbe-4382-846c-1f98e2b3f635', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:14:07,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2023-08-08 06:14:07,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.001) gunicorn-web stdout | 2023-08-08 06:14:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:07,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:07,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:07,190 [199] [DEBUG] [app] Starting request: urn:request:da1b3dc3-238e-42aa-9131-007a8e195611 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:14:07,190 [199] [DEBUG] [app] Ending request: urn:request:da1b3dc3-238e-42aa-9131-007a8e195611 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:da1b3dc3-238e-42aa-9131-007a8e195611', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:14:07,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:14:07,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:07,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:07,191 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:14:07,192 [198] [DEBUG] [app] Starting request: urn:request:d5e8cc17-fe6a-40ae-af37-3a87b0c03309 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:14:07,193 [198] [DEBUG] [app] Ending request: urn:request:d5e8cc17-fe6a-40ae-af37-3a87b0c03309 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d5e8cc17-fe6a-40ae-af37-3a87b0c03309', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:14:07,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:14:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:07,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:14:07,196 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:14:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:14:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:14:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:37.197412+00:00 (in 29.999376 seconds) buildlogsarchiver stdout | 2023-08-08 06:14:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:37 UTC)" (scheduled at 2023-08-08 06:14:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:14:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 14, 7, 198452), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:14:07,201 [199] [DEBUG] [app] Ending request: urn:request:661eddef-7f30-457b-baa5-49e628c02357 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:661eddef-7f30-457b-baa5-49e628c02357', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:14:07,201 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:14:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.023 47 0.024) gunicorn-web stdout | 2023-08-08 06:14:07,201 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:14:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:14:07,202 [197] [DEBUG] [app] Ending request: urn:request:e44d8e14-15b5-4e0b-ae4f-7d55e5669b56 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e44d8e14-15b5-4e0b-ae4f-7d55e5669b56', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:14:07,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:14:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:14:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:14:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.023 47 0.023) buildlogsarchiver stdout | 2023-08-08 06:14:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:14:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:14:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:14:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:14:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:14:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:14:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:19.417660+00:00 (in 9.999489 seconds) gcworker stdout | 2023-08-08 06:14:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:19 UTC)" (scheduled at 2023-08-08 06:14:09.417660+00:00) gcworker stdout | 2023-08-08 06:14:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265649418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:14:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:14:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:14:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:14:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:14:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:19.187230+00:00 (in 9.393038 seconds) repositorygcworker stdout | 2023-08-08 06:14:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:09 UTC)" (scheduled at 2023-08-08 06:14:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:14:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:14:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 9, 795235), True, datetime.datetime(2023, 8, 8, 6, 14, 9, 795235), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:14:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:14:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:14:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:14:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:14:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:22.293757+00:00 (in 9.999508 seconds) chunkcleanupworker stdout | 2023-08-08 06:14:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:22 UTC)" (scheduled at 2023-08-08 06:14:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:14:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:14:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 12, 294749), True, datetime.datetime(2023, 8, 8, 6, 14, 12, 294749), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:14:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:14:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:14:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:22 UTC)" executed successfully builder stdout | 2023-08-08 06:14:13,048 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:14:13,056 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:14:13,056 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:14:13,061 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:14:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:14:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:54.284093+00:00 (in 41.186217 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:14:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:13 UTC)" (scheduled at 2023-08-08 06:14:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:14:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:14:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 13, 98328), True, datetime.datetime(2023, 8, 8, 6, 14, 13, 98328), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:14:13,105 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:14:13,105 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:14:13,105 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:14:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:14:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:25.371682+00:00 (in 9.999522 seconds) notificationworker stdout | 2023-08-08 06:14:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:25 UTC)" (scheduled at 2023-08-08 06:14:15.371682+00:00) notificationworker stdout | 2023-08-08 06:14:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:14:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 15, 372535), True, datetime.datetime(2023, 8, 8, 6, 14, 15, 372535), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:14:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:14:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:14:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:14:18,241 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} repositorygcworker stdout | 2023-08-08 06:14:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:14:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:04.187429+00:00 (in 44.999615 seconds) repositorygcworker stdout | 2023-08-08 06:14:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:19:19 UTC)" (scheduled at 2023-08-08 06:14:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:14:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 14, 19, 188267), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:14:19,195 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 14, 19, 188267), True, datetime.datetime(2023, 8, 8, 6, 14, 19, 188267), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:14:19,197 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 14, 19, 188267), True, datetime.datetime(2023, 8, 8, 6, 14, 19, 188267), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 14, 19, 188267), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:14:19,198 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:14:19,199 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:19:19 UTC)" executed successfully gcworker stdout | 2023-08-08 06:14:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:14:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:29.417660+00:00 (in 9.999489 seconds) gcworker stdout | 2023-08-08 06:14:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:29 UTC)" (scheduled at 2023-08-08 06:14:19.417660+00:00) gcworker stdout | 2023-08-08 06:14:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265659418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:14:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:14:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:14:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:29 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:14:19,565 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:14:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:14:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:26.890741+00:00 (in 7.202913 seconds) securityworker stdout | 2023-08-08 06:14:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:49 UTC)" (scheduled at 2023-08-08 06:14:19.687363+00:00) securityworker stdout | 2023-08-08 06:14:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:14:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:14:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:14:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:14:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:14:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:14:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 9, 19, 691865), 1, 2]) securityworker stdout | 2023-08-08 06:14:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 9, 19, 691865), 1, 2]) securityworker stdout | 2023-08-08 06:14:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:14:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:14:20,582 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:14:21,033 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:14:22,179 [197] [DEBUG] [app] Starting request: urn:request:8e283594-e07d-4bec-9cd5-d2f6c97d104e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:14:22,180 [198] [DEBUG] [app] Starting request: urn:request:0f9f6498-e9e9-4b70-b69c-bf80bbfbc8da (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:14:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:22,182 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:22,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:14:22,185 [207] [DEBUG] [app] Starting request: urn:request:0072d305-1923-4b1a-9f0c-f07e61c34d01 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:14:22,186 [207] [DEBUG] [app] Ending request: urn:request:0072d305-1923-4b1a-9f0c-f07e61c34d01 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0072d305-1923-4b1a-9f0c-f07e61c34d01', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:14:22,186 [208] [DEBUG] [app] Starting request: urn:request:ffb2db59-9474-4b8c-98b3-878909740f85 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:14:22,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:14:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:14:22,186 [208] [DEBUG] [app] Ending request: urn:request:ffb2db59-9474-4b8c-98b3-878909740f85 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ffb2db59-9474-4b8c-98b3-878909740f85', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:14:22,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:14:22,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:22,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:22,191 [198] [DEBUG] [app] Starting request: urn:request:2e4489ec-a009-48ce-93c7-19433f110990 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:14:22,191 [198] [DEBUG] [app] Ending request: urn:request:2e4489ec-a009-48ce-93c7-19433f110990 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2e4489ec-a009-48ce-93c7-19433f110990', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:14:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:14:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:22,192 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:14:22,193 [200] [DEBUG] [app] Starting request: urn:request:54da2f5d-c431-43d7-ace9-1ec95723fa9d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:14:22,194 [200] [DEBUG] [app] Ending request: urn:request:54da2f5d-c431-43d7-ace9-1ec95723fa9d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:54da2f5d-c431-43d7-ace9-1ec95723fa9d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:14:22,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:14:22,194 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:22,195 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:14:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:14:22,208 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:14:22,233 [197] [DEBUG] [app] Ending request: urn:request:8e283594-e07d-4bec-9cd5-d2f6c97d104e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8e283594-e07d-4bec-9cd5-d2f6c97d104e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:14:22,233 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:14:22,233 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:14:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:14:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.055 47 0.054) gunicorn-web stdout | 2023-08-08 06:14:22,258 [198] [DEBUG] [app] Ending request: urn:request:0f9f6498-e9e9-4b70-b69c-bf80bbfbc8da (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0f9f6498-e9e9-4b70-b69c-bf80bbfbc8da', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:14:22,258 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:14:22,258 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:14:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:14:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.078 47 0.079) chunkcleanupworker stdout | 2023-08-08 06:14:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:14:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:28.874906+00:00 (in 6.580613 seconds) chunkcleanupworker stdout | 2023-08-08 06:14:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:32 UTC)" (scheduled at 2023-08-08 06:14:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:14:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:14:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 22, 294817), True, datetime.datetime(2023, 8, 8, 6, 14, 22, 294817), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:14:22,336 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:14:22,336 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:14:22,337 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:32 UTC)" executed successfully builder stdout | 2023-08-08 06:14:23,070 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:14:23,079 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:14:23,079 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:14:23,084 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:14:23,799 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:14:25,093 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:14:25,096 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:14:25,099 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:14:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:14:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:35.371682+00:00 (in 9.999506 seconds) notificationworker stdout | 2023-08-08 06:14:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:35 UTC)" (scheduled at 2023-08-08 06:14:25.371682+00:00) notificationworker stdout | 2023-08-08 06:14:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:14:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 25, 372530), True, datetime.datetime(2023, 8, 8, 6, 14, 25, 372530), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:14:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:14:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:14:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:14:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:14:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:47.398551+00:00 (in 21.883701 seconds) exportactionlogsworker stdout | 2023-08-08 06:14:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:25 UTC)" (scheduled at 2023-08-08 06:14:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:14:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:14:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:14:25,550 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:14:25,834 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:14:26,074 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:14:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:14:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:49.687363+00:00 (in 22.796082 seconds) securityworker stdout | 2023-08-08 06:14:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:56 UTC)" (scheduled at 2023-08-08 06:14:26.890741+00:00) securityworker stdout | 2023-08-08 06:14:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:14:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:14:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:14:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:14:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:14:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:14:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:14:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:14:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:14:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:14:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 9, 26, 896544), 1, 2]) securityworker stdout | 2023-08-08 06:14:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 9, 26, 896544), 1, 2]) securityworker stdout | 2023-08-08 06:14:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:14:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:14:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:14:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:14:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:14:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:14:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:14:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:14:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:14:27,041 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:14:27,508 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:14:27,607 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:14:27,657 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} queuecleanupworker stdout | 2023-08-08 06:14:27,671 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:14:27,685 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} repositoryactioncounter stdout | 2023-08-08 06:14:27,689 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gunicorn-web stdout | 2023-08-08 06:14:27,692 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} chunkcleanupworker stdout | 2023-08-08 06:14:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:14:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:32.293757+00:00 (in 3.418325 seconds) chunkcleanupworker stdout | 2023-08-08 06:14:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:28 UTC)" (scheduled at 2023-08-08 06:14:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:14:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:14:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:28 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:14:29,045 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:14:29,188 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:14:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:14:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:39.417660+00:00 (in 9.999490 seconds) gcworker stdout | 2023-08-08 06:14:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:39 UTC)" (scheduled at 2023-08-08 06:14:29.417660+00:00) gcworker stdout | 2023-08-08 06:14:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265669418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:14:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:14:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:14:29,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:14:30,264 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:14:31,160 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:14:31,223 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:14:31,718 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:14:32,170 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:14:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:14:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:42.293757+00:00 (in 9.999467 seconds) chunkcleanupworker stdout | 2023-08-08 06:14:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:42 UTC)" (scheduled at 2023-08-08 06:14:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:14:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:14:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 32, 294765), True, datetime.datetime(2023, 8, 8, 6, 14, 32, 294765), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:14:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:14:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:14:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:42 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:14:32,537 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:14:32,830 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:14:33,094 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:14:33,104 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:14:33,104 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:14:33,109 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:14:33,384 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:14:33,389 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:14:33,392 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:14:33,396 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:14:33,399 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:14:33,404 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:14:33,413 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:14:33,416 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:14:33,419 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} notificationworker stdout | 2023-08-08 06:14:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:14:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:45.371682+00:00 (in 9.999482 seconds) notificationworker stdout | 2023-08-08 06:14:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:45 UTC)" (scheduled at 2023-08-08 06:14:35.371682+00:00) notificationworker stdout | 2023-08-08 06:14:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:14:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 35, 372666), True, datetime.datetime(2023, 8, 8, 6, 14, 35, 372666), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:14:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:14:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:14:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:14:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:14:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:39.579576+00:00 (in 2.999148 seconds) namespacegcworker stdout | 2023-08-08 06:14:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:36 UTC)" (scheduled at 2023-08-08 06:14:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:14:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:14:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:14:37,178 [197] [DEBUG] [app] Starting request: urn:request:df64f931-ad8c-4177-9ef2-273fcadbf624 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:14:37,179 [200] [DEBUG] [app] Starting request: urn:request:a570b5b8-6f6e-494d-9c06-c4ecec14e876 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:14:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:14:37,184 [210] [DEBUG] [app] Starting request: urn:request:003e256a-ce05-44f2-944e-b861a4467f3e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:14:37,185 [203] [DEBUG] [app] Starting request: urn:request:92f46f7a-9e49-43ab-98a3-a1e5b0451bfa (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:14:37,185 [210] [DEBUG] [app] Ending request: urn:request:003e256a-ce05-44f2-944e-b861a4467f3e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:003e256a-ce05-44f2-944e-b861a4467f3e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:14:37,185 [203] [DEBUG] [app] Ending request: urn:request:92f46f7a-9e49-43ab-98a3-a1e5b0451bfa (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:92f46f7a-9e49-43ab-98a3-a1e5b0451bfa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:14:37,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:14:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:14:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:14:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:37,191 [198] [DEBUG] [app] Starting request: urn:request:5a523dce-c570-48bc-98f9-4afcef932b43 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:14:37,191 [199] [DEBUG] [app] Starting request: urn:request:d785f3b9-cc4d-4e4e-8316-3180ae39767a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:14:37,191 [198] [DEBUG] [app] Ending request: urn:request:5a523dce-c570-48bc-98f9-4afcef932b43 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5a523dce-c570-48bc-98f9-4afcef932b43', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:14:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:14:37,191 [199] [DEBUG] [app] Ending request: urn:request:d785f3b9-cc4d-4e4e-8316-3180ae39767a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d785f3b9-cc4d-4e4e-8316-3180ae39767a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:14:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:14:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:14:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:14:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:14:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:14:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:14:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:07.197412+00:00 (in 29.999546 seconds) buildlogsarchiver stdout | 2023-08-08 06:14:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:07 UTC)" (scheduled at 2023-08-08 06:14:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:14:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 14, 37, 198388), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:14:37,202 [200] [DEBUG] [app] Ending request: urn:request:a570b5b8-6f6e-494d-9c06-c4ecec14e876 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a570b5b8-6f6e-494d-9c06-c4ecec14e876', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:14:37,202 [197] [DEBUG] [app] Ending request: urn:request:df64f931-ad8c-4177-9ef2-273fcadbf624 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:df64f931-ad8c-4177-9ef2-273fcadbf624', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:14:37,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:14:37,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:14:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:14:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:14:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:14:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:14:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:14:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:14:37,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:14:37,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:14:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:14:37,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:14:37,209 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:14:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:14:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:49.417660+00:00 (in 9.999486 seconds) gcworker stdout | 2023-08-08 06:14:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:49 UTC)" (scheduled at 2023-08-08 06:14:39.417660+00:00) gcworker stdout | 2023-08-08 06:14:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265679418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:14:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:14:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:14:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:14:39,579 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:14:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:54.393627+00:00 (in 14.813510 seconds) namespacegcworker stdout | 2023-08-08 06:14:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:19:39 UTC)" (scheduled at 2023-08-08 06:14:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:14:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 14, 39, 580464), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:14:39,587 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 14, 39, 580464), True, datetime.datetime(2023, 8, 8, 6, 14, 39, 580464), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:14:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 14, 39, 580464), True, datetime.datetime(2023, 8, 8, 6, 14, 39, 580464), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 14, 39, 580464), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:14:39,590 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:14:39,590 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:19:39 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:14:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:14:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:46.874674+00:00 (in 4.580425 seconds) chunkcleanupworker stdout | 2023-08-08 06:14:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:52 UTC)" (scheduled at 2023-08-08 06:14:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:14:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:14:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 42, 294744), True, datetime.datetime(2023, 8, 8, 6, 14, 42, 294744), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:14:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:14:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:14:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:52 UTC)" executed successfully builder stdout | 2023-08-08 06:14:43,119 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:14:43,128 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:14:43,128 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:14:43,133 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:14:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:14:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:55.371682+00:00 (in 9.999440 seconds) notificationworker stdout | 2023-08-08 06:14:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:55 UTC)" (scheduled at 2023-08-08 06:14:45.371682+00:00) notificationworker stdout | 2023-08-08 06:14:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:14:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 45, 372654), True, datetime.datetime(2023, 8, 8, 6, 14, 45, 372654), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:14:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:14:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:14:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:14:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:14:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:52.293757+00:00 (in 5.418586 seconds) chunkcleanupworker stdout | 2023-08-08 06:14:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:19:46 UTC)" (scheduled at 2023-08-08 06:14:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:14:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 14, 46, 875633), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:14:46,882 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 14, 46, 875633), True, datetime.datetime(2023, 8, 8, 6, 14, 46, 875633), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:14:46,884 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 14, 46, 875633), True, datetime.datetime(2023, 8, 8, 6, 14, 46, 875633), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 14, 46, 875633), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:14:46,885 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:14:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:19:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:14:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:14:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:25.514341+00:00 (in 38.115278 seconds) exportactionlogsworker stdout | 2023-08-08 06:14:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:47 UTC)" (scheduled at 2023-08-08 06:14:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:14:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:14:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 47, 399539), True, datetime.datetime(2023, 8, 8, 6, 14, 47, 399539), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:14:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:14:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:14:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:14:48,278 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:14:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:14:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:59.417660+00:00 (in 9.999478 seconds) gcworker stdout | 2023-08-08 06:14:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:59 UTC)" (scheduled at 2023-08-08 06:14:49.417660+00:00) gcworker stdout | 2023-08-08 06:14:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265689418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:14:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:14:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:14:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:14:59 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:14:49,577 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:14:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:14:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:14:56.890741+00:00 (in 7.202884 seconds) securityworker stdout | 2023-08-08 06:14:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:19 UTC)" (scheduled at 2023-08-08 06:14:49.687363+00:00) securityworker stdout | 2023-08-08 06:14:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:14:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:14:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:14:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:14:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:14:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:14:50,597 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:14:51,058 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:14:52,178 [198] [DEBUG] [app] Starting request: urn:request:117fdcef-e0a5-4e1c-9ed5-3f224a057e05 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:14:52,178 [199] [DEBUG] [app] Starting request: urn:request:ea4ce534-da14-45b2-af9c-8c77f7cf82a6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:14:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:52,182 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:14:52,184 [203] [DEBUG] [app] Starting request: urn:request:6e2635d3-7dd8-4cba-8e83-407553912371 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:14:52,184 [203] [DEBUG] [app] Ending request: urn:request:6e2635d3-7dd8-4cba-8e83-407553912371 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6e2635d3-7dd8-4cba-8e83-407553912371', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:14:52,184 [206] [DEBUG] [app] Starting request: urn:request:6ce04958-2ad8-47b4-b7b7-500e15a8c6e7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:14:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:14:52,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:14:52,185 [206] [DEBUG] [app] Ending request: urn:request:6ce04958-2ad8-47b4-b7b7-500e15a8c6e7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6ce04958-2ad8-47b4-b7b7-500e15a8c6e7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:14:52,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:14:52,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:52,186 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:52,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:14:52,188 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:52,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:14:52,189 [200] [DEBUG] [app] Starting request: urn:request:c8594c81-3f72-4b92-be05-c4c76aebf52a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:14:52,190 [200] [DEBUG] [app] Ending request: urn:request:c8594c81-3f72-4b92-be05-c4c76aebf52a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c8594c81-3f72-4b92-be05-c4c76aebf52a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:14:52,190 [197] [DEBUG] [app] Starting request: urn:request:4d203619-c5f7-4ed0-8b91-f03c2f6dec19 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:14:52,190 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:14:52,190 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:52,191 [197] [DEBUG] [app] Ending request: urn:request:4d203619-c5f7-4ed0-8b91-f03c2f6dec19 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4d203619-c5f7-4ed0-8b91-f03c2f6dec19', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:14:52,191 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:14:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:14:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:14:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:14:52,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:14:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:14:52,196 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:14:52,196 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:14:52,201 [198] [DEBUG] [app] Ending request: urn:request:117fdcef-e0a5-4e1c-9ed5-3f224a057e05 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:117fdcef-e0a5-4e1c-9ed5-3f224a057e05', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:14:52,201 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:14:52,201 [199] [DEBUG] [app] Ending request: urn:request:ea4ce534-da14-45b2-af9c-8c77f7cf82a6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ea4ce534-da14-45b2-af9c-8c77f7cf82a6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:14:52,201 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:14:52,201 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:14:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:14:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:14:52,201 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:14:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:14:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) chunkcleanupworker stdout | 2023-08-08 06:14:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:14:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:02.293757+00:00 (in 9.999497 seconds) chunkcleanupworker stdout | 2023-08-08 06:14:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:02 UTC)" (scheduled at 2023-08-08 06:14:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:14:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:14:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 52, 294787), True, datetime.datetime(2023, 8, 8, 6, 14, 52, 294787), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:14:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:14:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:14:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:02 UTC)" executed successfully builder stdout | 2023-08-08 06:14:53,138 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:14:53,147 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:14:53,147 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:14:53,152 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:14:53,809 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:14:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:14:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:13.097375+00:00 (in 18.812739 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:14:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:54 UTC)" (scheduled at 2023-08-08 06:14:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:14:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:14:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:14:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:14:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:36.579841+00:00 (in 42.185726 seconds) namespacegcworker stdout | 2023-08-08 06:14:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:54 UTC)" (scheduled at 2023-08-08 06:14:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:14:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:14:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 54, 394480), True, datetime.datetime(2023, 8, 8, 6, 14, 54, 394480), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:14:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:14:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:14:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:15:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:14:55,102 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:14:55,105 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:14:55,109 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:14:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:14:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:02.178530+00:00 (in 6.806328 seconds) notificationworker stdout | 2023-08-08 06:14:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:05 UTC)" (scheduled at 2023-08-08 06:14:55.371682+00:00) notificationworker stdout | 2023-08-08 06:14:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:14:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 14, 55, 372591), True, datetime.datetime(2023, 8, 8, 6, 14, 55, 372591), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:14:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:14:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:14:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:14:55,587 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:14:55,871 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:14:56,106 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:14:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:14:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:19.687363+00:00 (in 22.796126 seconds) securityworker stdout | 2023-08-08 06:14:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:26 UTC)" (scheduled at 2023-08-08 06:14:56.890741+00:00) securityworker stdout | 2023-08-08 06:14:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:14:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:14:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:14:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:14:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:14:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:14:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:14:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:14:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:14:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:14:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 9, 56, 896386), 1, 2]) securityworker stdout | 2023-08-08 06:14:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 9, 56, 896386), 1, 2]) securityworker stdout | 2023-08-08 06:14:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:14:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:14:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:14:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:14:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:14:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:14:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:14:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:14:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:14:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:14:57,078 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:14:57,517 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:14:57,617 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:14:57,670 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} queuecleanupworker stdout | 2023-08-08 06:14:57,687 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:14:57,697 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:14:57,703 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:14:57,725 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} buildlogsarchiver stdout | 2023-08-08 06:14:59,077 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:14:59,226 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:14:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:14:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:09.417660+00:00 (in 9.999493 seconds) gcworker stdout | 2023-08-08 06:14:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:09 UTC)" (scheduled at 2023-08-08 06:14:59.417660+00:00) gcworker stdout | 2023-08-08 06:14:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265699418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:14:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:14:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:14:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:15:00,292 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:15:01,177 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:15:01,261 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:15:01,757 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:15:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:15:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:05.371682+00:00 (in 3.192554 seconds) notificationworker stdout | 2023-08-08 06:15:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:02 UTC)" (scheduled at 2023-08-08 06:15:02.178530+00:00) notificationworker stdout | 2023-08-08 06:15:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:15:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:15:02,191 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:15:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:15:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:12.293757+00:00 (in 9.999494 seconds) chunkcleanupworker stdout | 2023-08-08 06:15:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:12 UTC)" (scheduled at 2023-08-08 06:15:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:15:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:15:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 2, 294858), True, datetime.datetime(2023, 8, 8, 6, 15, 2, 294858), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:15:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:15:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:15:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:12 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:15:02,553 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:15:02,860 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:15:03,162 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:15:03,170 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:15:03,171 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:15:03,175 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:15:03,392 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:15:03,401 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:15:03,404 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:15:03,409 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:15:03,412 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:15:03,415 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:15:03,424 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:15:03,427 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:15:03,430 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:15:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:15:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:09.793658+00:00 (in 5.605701 seconds) repositorygcworker stdout | 2023-08-08 06:15:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:04 UTC)" (scheduled at 2023-08-08 06:15:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:15:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:15:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:15:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:15:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:15.371682+00:00 (in 9.999415 seconds) notificationworker stdout | 2023-08-08 06:15:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:15 UTC)" (scheduled at 2023-08-08 06:15:05.371682+00:00) notificationworker stdout | 2023-08-08 06:15:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:15:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 5, 372709), True, datetime.datetime(2023, 8, 8, 6, 15, 5, 372709), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:15:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:15:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:15:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:15:07,179 [197] [DEBUG] [app] Starting request: urn:request:fb6ab869-3f1a-451a-81ea-06a6d13ed1e0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:15:07,179 [200] [DEBUG] [app] Starting request: urn:request:31abaec0-27ae-4022-be50-174ed8f3a08c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:15:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:15:07,184 [210] [DEBUG] [app] Starting request: urn:request:2fbd160a-296f-4b88-8b13-a87510bc1964 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:15:07,185 [203] [DEBUG] [app] Starting request: urn:request:1dfe706b-8bb0-4963-b1ac-05d06239733f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:15:07,185 [210] [DEBUG] [app] Ending request: urn:request:2fbd160a-296f-4b88-8b13-a87510bc1964 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2fbd160a-296f-4b88-8b13-a87510bc1964', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:15:07,185 [203] [DEBUG] [app] Ending request: urn:request:1dfe706b-8bb0-4963-b1ac-05d06239733f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1dfe706b-8bb0-4963-b1ac-05d06239733f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:15:07,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:15:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:07,191 [197] [DEBUG] [app] Starting request: urn:request:49c572b5-47b2-47ac-9e33-b30db58378ec (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:15:07,191 [197] [DEBUG] [app] Ending request: urn:request:49c572b5-47b2-47ac-9e33-b30db58378ec (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:49c572b5-47b2-47ac-9e33-b30db58378ec', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:07,191 [199] [DEBUG] [app] Starting request: urn:request:d18505c8-3ddc-47fe-b174-d855f1ef7454 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:15:07,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:15:07,192 [199] [DEBUG] [app] Ending request: urn:request:d18505c8-3ddc-47fe-b174-d855f1ef7454 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d18505c8-3ddc-47fe-b174-d855f1ef7454', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:15:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:15:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:15:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:15:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:15:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:15:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:37.197412+00:00 (in 29.999318 seconds) buildlogsarchiver stdout | 2023-08-08 06:15:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:37 UTC)" (scheduled at 2023-08-08 06:15:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:15:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 15, 7, 198715), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:15:07,202 [197] [DEBUG] [app] Ending request: urn:request:fb6ab869-3f1a-451a-81ea-06a6d13ed1e0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fb6ab869-3f1a-451a-81ea-06a6d13ed1e0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:15:07,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:15:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:15:07,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:15:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:15:07,203 [200] [DEBUG] [app] Ending request: urn:request:31abaec0-27ae-4022-be50-174ed8f3a08c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:31abaec0-27ae-4022-be50-174ed8f3a08c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:15:07,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:15:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:15:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:15:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:15:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:15:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:15:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:15:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:15:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:15:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:15:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:19.417660+00:00 (in 9.999489 seconds) gcworker stdout | 2023-08-08 06:15:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:19 UTC)" (scheduled at 2023-08-08 06:15:09.417660+00:00) gcworker stdout | 2023-08-08 06:15:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265709418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:15:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:15:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:15:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:15:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:15:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:04.187429+00:00 (in 54.393235 seconds) repositorygcworker stdout | 2023-08-08 06:15:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:09 UTC)" (scheduled at 2023-08-08 06:15:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:15:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:15:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 9, 795457), True, datetime.datetime(2023, 8, 8, 6, 15, 9, 795457), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:15:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:15:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:15:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:15:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:15:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:22.293757+00:00 (in 9.999505 seconds) chunkcleanupworker stdout | 2023-08-08 06:15:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:22 UTC)" (scheduled at 2023-08-08 06:15:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:15:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:15:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 12, 294943), True, datetime.datetime(2023, 8, 8, 6, 15, 12, 294943), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:15:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:15:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:15:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:15:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:15:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:54.284093+00:00 (in 41.186135 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:15:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:13 UTC)" (scheduled at 2023-08-08 06:15:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:15:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:15:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 13, 98347), True, datetime.datetime(2023, 8, 8, 6, 15, 13, 98347), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:15:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:15:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:15:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:13 UTC)" executed successfully builder stdout | 2023-08-08 06:15:13,185 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:15:13,193 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:15:13,194 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:15:13,198 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:15:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:15:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:25.371682+00:00 (in 9.999484 seconds) notificationworker stdout | 2023-08-08 06:15:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:25 UTC)" (scheduled at 2023-08-08 06:15:15.371682+00:00) notificationworker stdout | 2023-08-08 06:15:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:15:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 15, 372534), True, datetime.datetime(2023, 8, 8, 6, 15, 15, 372534), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:15:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:15:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:15:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:15:18,315 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:15:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:15:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:29.417660+00:00 (in 9.999445 seconds) gcworker stdout | 2023-08-08 06:15:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:29 UTC)" (scheduled at 2023-08-08 06:15:19.417660+00:00) gcworker stdout | 2023-08-08 06:15:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265719418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:15:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:15:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:15:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:29 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:15:19,593 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:15:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:15:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:26.890741+00:00 (in 7.202868 seconds) securityworker stdout | 2023-08-08 06:15:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:49 UTC)" (scheduled at 2023-08-08 06:15:19.687363+00:00) securityworker stdout | 2023-08-08 06:15:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:15:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:15:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:15:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:15:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:15:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:15:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 10, 19, 691942), 1, 2]) securityworker stdout | 2023-08-08 06:15:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 10, 19, 691942), 1, 2]) securityworker stdout | 2023-08-08 06:15:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:15:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:15:20,625 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:15:21,095 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:15:22,179 [197] [DEBUG] [app] Starting request: urn:request:00ce5dd5-9678-44ee-8324-6fe083f12b2f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:15:22,179 [200] [DEBUG] [app] Starting request: urn:request:41c17461-b719-43a7-9824-aa16a2d286ce (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:15:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:22,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:15:22,185 [203] [DEBUG] [app] Starting request: urn:request:3e5b7bec-dbfe-449b-a326-37c6f669a2d3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:15:22,186 [203] [DEBUG] [app] Ending request: urn:request:3e5b7bec-dbfe-449b-a326-37c6f669a2d3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3e5b7bec-dbfe-449b-a326-37c6f669a2d3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:15:22,186 [207] [DEBUG] [app] Starting request: urn:request:bd062fc7-1c7a-49ab-bb24-a56d1266f23c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:15:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:15:22,186 [207] [DEBUG] [app] Ending request: urn:request:bd062fc7-1c7a-49ab-bb24-a56d1266f23c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:bd062fc7-1c7a-49ab-bb24-a56d1266f23c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:15:22,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:22,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:22,192 [199] [DEBUG] [app] Starting request: urn:request:eed28742-b4fc-4434-bd4f-51d1919910ce (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:15:22,192 [197] [DEBUG] [app] Starting request: urn:request:e6ca67fd-f65d-430c-a9d2-13c8e8a52b4b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:15:22,193 [199] [DEBUG] [app] Ending request: urn:request:eed28742-b4fc-4434-bd4f-51d1919910ce (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:eed28742-b4fc-4434-bd4f-51d1919910ce', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:22,193 [197] [DEBUG] [app] Ending request: urn:request:e6ca67fd-f65d-430c-a9d2-13c8e8a52b4b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e6ca67fd-f65d-430c-a9d2-13c8e8a52b4b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:15:22,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:22,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:22,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:15:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:15:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:15:22,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:15:22,204 [197] [DEBUG] [app] Ending request: urn:request:00ce5dd5-9678-44ee-8324-6fe083f12b2f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:00ce5dd5-9678-44ee-8324-6fe083f12b2f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:15:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:15:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:15:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:15:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:15:22,205 [200] [DEBUG] [app] Ending request: urn:request:41c17461-b719-43a7-9824-aa16a2d286ce (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:41c17461-b719-43a7-9824-aa16a2d286ce', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:15:22,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:15:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:15:22,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:15:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:15:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:15:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:28.874906+00:00 (in 6.580664 seconds) chunkcleanupworker stdout | 2023-08-08 06:15:22,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:32 UTC)" (scheduled at 2023-08-08 06:15:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:15:22,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:15:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 22, 295152), True, datetime.datetime(2023, 8, 8, 6, 15, 22, 295152), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:15:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:15:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:15:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:32 UTC)" executed successfully builder stdout | 2023-08-08 06:15:23,208 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:15:23,217 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:15:23,218 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:15:23,222 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:15:23,825 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:15:25,110 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:15:25,113 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:15:25,116 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:15:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:15:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:35.371682+00:00 (in 9.999506 seconds) notificationworker stdout | 2023-08-08 06:15:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:35 UTC)" (scheduled at 2023-08-08 06:15:25.371682+00:00) notificationworker stdout | 2023-08-08 06:15:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:15:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 25, 372530), True, datetime.datetime(2023, 8, 8, 6, 15, 25, 372530), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:15:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:15:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:15:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:15:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:15:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:47.398551+00:00 (in 21.883685 seconds) exportactionlogsworker stdout | 2023-08-08 06:15:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:25 UTC)" (scheduled at 2023-08-08 06:15:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:15:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:15:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:15:25,624 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:15:25,908 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:15:26,117 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:15:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:15:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:49.687363+00:00 (in 22.796118 seconds) securityworker stdout | 2023-08-08 06:15:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:56 UTC)" (scheduled at 2023-08-08 06:15:26.890741+00:00) securityworker stdout | 2023-08-08 06:15:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:15:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:15:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:15:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:15:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:15:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:15:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:15:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:15:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:15:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:15:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 10, 26, 896336), 1, 2]) securityworker stdout | 2023-08-08 06:15:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 10, 26, 896336), 1, 2]) securityworker stdout | 2023-08-08 06:15:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:15:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:15:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:15:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:15:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:15:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:15:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:15:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:15:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:15:27,089 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:15:27,525 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:15:27,627 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:15:27,683 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:15:27,708 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:15:27,714 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} queuecleanupworker stdout | 2023-08-08 06:15:27,722 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:15:27,762 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} chunkcleanupworker stdout | 2023-08-08 06:15:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:15:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:32.293757+00:00 (in 3.418315 seconds) chunkcleanupworker stdout | 2023-08-08 06:15:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:28 UTC)" (scheduled at 2023-08-08 06:15:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:15:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:15:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:28 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:15:29,086 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:15:29,262 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:15:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:15:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:39.417660+00:00 (in 9.999448 seconds) gcworker stdout | 2023-08-08 06:15:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:39 UTC)" (scheduled at 2023-08-08 06:15:29.417660+00:00) gcworker stdout | 2023-08-08 06:15:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265729418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:15:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:15:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:15:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:15:30,321 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:15:31,211 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:15:31,293 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:15:31,764 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:15:32,229 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:15:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:15:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:42.293757+00:00 (in 9.999343 seconds) chunkcleanupworker stdout | 2023-08-08 06:15:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:42 UTC)" (scheduled at 2023-08-08 06:15:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:15:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:15:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 32, 294991), True, datetime.datetime(2023, 8, 8, 6, 15, 32, 294991), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:15:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:15:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:15:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:42 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:15:32,590 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:15:32,887 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:15:33,232 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:15:33,241 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:15:33,241 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:15:33,245 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:15:33,402 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:15:33,412 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:15:33,415 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:15:33,421 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:15:33,426 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:15:33,429 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:15:33,432 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:15:33,438 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:15:33,441 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} notificationworker stdout | 2023-08-08 06:15:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:15:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:45.371682+00:00 (in 9.999485 seconds) notificationworker stdout | 2023-08-08 06:15:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:45 UTC)" (scheduled at 2023-08-08 06:15:35.371682+00:00) notificationworker stdout | 2023-08-08 06:15:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:15:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 35, 372567), True, datetime.datetime(2023, 8, 8, 6, 15, 35, 372567), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:15:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:15:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:15:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:15:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:15:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:54.393627+00:00 (in 17.813271 seconds) namespacegcworker stdout | 2023-08-08 06:15:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:36 UTC)" (scheduled at 2023-08-08 06:15:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:15:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:15:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:15:37,179 [197] [DEBUG] [app] Starting request: urn:request:3b5578b4-c690-4977-893a-77484dd152ee (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:15:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:37,181 [200] [DEBUG] [app] Starting request: urn:request:cd6c09ea-61d0-401f-92dd-c8a78d6ede64 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:15:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:15:37,185 [204] [DEBUG] [app] Starting request: urn:request:8a578300-4b53-4fc5-8905-61f270e854d2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:15:37,186 [208] [DEBUG] [app] Starting request: urn:request:6c96d400-9c44-49a5-b4cf-2a6741332c17 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:15:37,186 [204] [DEBUG] [app] Ending request: urn:request:8a578300-4b53-4fc5-8905-61f270e854d2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8a578300-4b53-4fc5-8905-61f270e854d2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:15:37,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:15:37,186 [208] [DEBUG] [app] Ending request: urn:request:6c96d400-9c44-49a5-b4cf-2a6741332c17 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6c96d400-9c44-49a5-b4cf-2a6741332c17', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:15:37,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:15:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:37,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:37,191 [198] [DEBUG] [app] Starting request: urn:request:1db59c80-d381-4a9c-aa4a-db1b70e2d03e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:15:37,192 [199] [DEBUG] [app] Starting request: urn:request:0da51bd1-77dc-48a3-ac79-7a624cf22f23 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:15:37,192 [198] [DEBUG] [app] Ending request: urn:request:1db59c80-d381-4a9c-aa4a-db1b70e2d03e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1db59c80-d381-4a9c-aa4a-db1b70e2d03e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:37,192 [199] [DEBUG] [app] Ending request: urn:request:0da51bd1-77dc-48a3-ac79-7a624cf22f23 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0da51bd1-77dc-48a3-ac79-7a624cf22f23', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:15:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:15:37,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:15:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:15:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:15:37,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:15:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:07.197412+00:00 (in 29.999349 seconds) buildlogsarchiver stdout | 2023-08-08 06:15:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:07 UTC)" (scheduled at 2023-08-08 06:15:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:15:37,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:15:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 15, 37, 198472), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:15:37,203 [197] [DEBUG] [app] Ending request: urn:request:3b5578b4-c690-4977-893a-77484dd152ee (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3b5578b4-c690-4977-893a-77484dd152ee', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:15:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:15:37,203 [200] [DEBUG] [app] Ending request: urn:request:cd6c09ea-61d0-401f-92dd-c8a78d6ede64 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cd6c09ea-61d0-401f-92dd-c8a78d6ede64', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:15:37,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:15:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:15:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:15:37,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:15:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:15:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:15:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:15:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:15:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:15:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:15:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:15:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:15:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:15:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:49.417660+00:00 (in 9.999396 seconds) gcworker stdout | 2023-08-08 06:15:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:49 UTC)" (scheduled at 2023-08-08 06:15:39.417660+00:00) gcworker stdout | 2023-08-08 06:15:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265739418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:15:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:15:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:15:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:15:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:15:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:52.293757+00:00 (in 9.999449 seconds) chunkcleanupworker stdout | 2023-08-08 06:15:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:52 UTC)" (scheduled at 2023-08-08 06:15:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:15:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:15:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 42, 294823), True, datetime.datetime(2023, 8, 8, 6, 15, 42, 294823), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:15:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:15:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:15:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:52 UTC)" executed successfully builder stdout | 2023-08-08 06:15:43,254 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:15:43,263 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:15:43,263 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:15:43,267 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:15:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:15:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:55.371682+00:00 (in 9.999517 seconds) notificationworker stdout | 2023-08-08 06:15:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:55 UTC)" (scheduled at 2023-08-08 06:15:45.371682+00:00) notificationworker stdout | 2023-08-08 06:15:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:15:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 45, 372679), True, datetime.datetime(2023, 8, 8, 6, 15, 45, 372679), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:15:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:15:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:15:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:15:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:15:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:25.514341+00:00 (in 38.115249 seconds) exportactionlogsworker stdout | 2023-08-08 06:15:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:47 UTC)" (scheduled at 2023-08-08 06:15:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:15:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:15:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 47, 399587), True, datetime.datetime(2023, 8, 8, 6, 15, 47, 399587), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:15:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:15:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:15:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:15:48,341 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:15:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:15:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:59.417660+00:00 (in 9.999473 seconds) gcworker stdout | 2023-08-08 06:15:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:59 UTC)" (scheduled at 2023-08-08 06:15:49.417660+00:00) gcworker stdout | 2023-08-08 06:15:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265749418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:15:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:15:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:15:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:15:59 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:15:49,630 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:15:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:15:49,687 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:15:56.890741+00:00 (in 7.202923 seconds) securityworker stdout | 2023-08-08 06:15:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:19 UTC)" (scheduled at 2023-08-08 06:15:49.687363+00:00) securityworker stdout | 2023-08-08 06:15:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:15:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:15:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:15:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:15:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:15:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:15:50,655 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:15:51,105 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:15:52,179 [200] [DEBUG] [app] Starting request: urn:request:7ca32812-7997-4370-a1fe-37523f142fb2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:15:52,179 [198] [DEBUG] [app] Starting request: urn:request:a29a4e26-2ffb-43ff-9258-202a734a5438 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:15:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:52,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:15:52,184 [203] [DEBUG] [app] Starting request: urn:request:3d0903e6-8573-45dc-a90a-79462fc4928c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:15:52,184 [208] [DEBUG] [app] Starting request: urn:request:b2f99b0f-9a09-4f80-bc86-2dd9ebc39df2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:15:52,185 [203] [DEBUG] [app] Ending request: urn:request:3d0903e6-8573-45dc-a90a-79462fc4928c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3d0903e6-8573-45dc-a90a-79462fc4928c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:15:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:15:52,185 [208] [DEBUG] [app] Ending request: urn:request:b2f99b0f-9a09-4f80-bc86-2dd9ebc39df2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b2f99b0f-9a09-4f80-bc86-2dd9ebc39df2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:15:52,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:15:52,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:52,186 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:15:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:52,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:15:52,190 [199] [DEBUG] [app] Starting request: urn:request:52857ac0-3712-442b-a246-565abcf19285 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:15:52,190 [197] [DEBUG] [app] Starting request: urn:request:3a42ade7-7fdc-4e62-9c15-6c57b8388884 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:15:52,190 [199] [DEBUG] [app] Ending request: urn:request:52857ac0-3712-442b-a246-565abcf19285 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:52857ac0-3712-442b-a246-565abcf19285', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:52,190 [197] [DEBUG] [app] Ending request: urn:request:3a42ade7-7fdc-4e62-9c15-6c57b8388884 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3a42ade7-7fdc-4e62-9c15-6c57b8388884', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:15:52,190 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:15:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:15:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:15:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:15:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:15:52,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:15:52,191 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:15:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:15:52,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:15:52,196 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:15:52,201 [200] [DEBUG] [app] Ending request: urn:request:7ca32812-7997-4370-a1fe-37523f142fb2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7ca32812-7997-4370-a1fe-37523f142fb2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:15:52,201 [198] [DEBUG] [app] Ending request: urn:request:a29a4e26-2ffb-43ff-9258-202a734a5438 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a29a4e26-2ffb-43ff-9258-202a734a5438', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:15:52,201 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:15:52,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:15:52,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:15:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:15:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:15:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:15:52,202 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:15:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:15:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:15:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:02.293757+00:00 (in 9.999353 seconds) chunkcleanupworker stdout | 2023-08-08 06:15:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:02 UTC)" (scheduled at 2023-08-08 06:15:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:15:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:15:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 52, 294877), True, datetime.datetime(2023, 8, 8, 6, 15, 52, 294877), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:15:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:15:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:15:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:02 UTC)" executed successfully builder stdout | 2023-08-08 06:15:53,277 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:15:53,286 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:15:53,287 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:15:53,291 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:15:53,833 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:15:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:15:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:13.097375+00:00 (in 18.812730 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:15:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:54 UTC)" (scheduled at 2023-08-08 06:15:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:15:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:15:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:15:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:15:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:36.579841+00:00 (in 42.185694 seconds) namespacegcworker stdout | 2023-08-08 06:15:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:54 UTC)" (scheduled at 2023-08-08 06:15:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:15:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:15:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 54, 394511), True, datetime.datetime(2023, 8, 8, 6, 15, 54, 394511), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:15:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:15:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:15:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:16:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:15:55,121 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:15:55,124 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:15:55,128 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:15:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:15:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:02.178530+00:00 (in 6.806280 seconds) notificationworker stdout | 2023-08-08 06:15:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:05 UTC)" (scheduled at 2023-08-08 06:15:55.371682+00:00) notificationworker stdout | 2023-08-08 06:15:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:15:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 15, 55, 372664), True, datetime.datetime(2023, 8, 8, 6, 15, 55, 372664), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:15:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:15:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:15:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:15:55,641 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:15:55,945 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:15:56,155 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:15:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:15:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:19.687363+00:00 (in 22.796096 seconds) securityworker stdout | 2023-08-08 06:15:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:26 UTC)" (scheduled at 2023-08-08 06:15:56.890741+00:00) securityworker stdout | 2023-08-08 06:15:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:15:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:15:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:15:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:15:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:15:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:15:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:15:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:15:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:15:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:15:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 10, 56, 896481), 1, 2]) securityworker stdout | 2023-08-08 06:15:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 10, 56, 896481), 1, 2]) securityworker stdout | 2023-08-08 06:15:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:15:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:15:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:15:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:15:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:15:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:15:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:15:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:15:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:15:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:15:57,097 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:15:57,533 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:15:57,637 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:15:57,694 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:15:57,723 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:15:57,728 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} queuecleanupworker stdout | 2023-08-08 06:15:57,759 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:15:57,798 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} buildlogsarchiver stdout | 2023-08-08 06:15:59,115 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:15:59,270 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:15:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:15:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:09.417660+00:00 (in 9.999471 seconds) gcworker stdout | 2023-08-08 06:15:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:09 UTC)" (scheduled at 2023-08-08 06:15:59.417660+00:00) gcworker stdout | 2023-08-08 06:15:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265759418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:15:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:15:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:15:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:16:00,332 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:16:01,229 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:16:01,315 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:16:01,801 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:16:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:16:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:05.371682+00:00 (in 3.192621 seconds) notificationworker stdout | 2023-08-08 06:16:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:02 UTC)" (scheduled at 2023-08-08 06:16:02.178530+00:00) notificationworker stdout | 2023-08-08 06:16:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:16:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:16:02,248 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:16:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:16:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:12.293757+00:00 (in 9.999488 seconds) chunkcleanupworker stdout | 2023-08-08 06:16:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:12 UTC)" (scheduled at 2023-08-08 06:16:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:16:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:16:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 2, 294812), True, datetime.datetime(2023, 8, 8, 6, 16, 2, 294812), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:16:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:16:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:16:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:12 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:16:02,598 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:16:02,923 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:16:03,301 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:16:03,310 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:16:03,310 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:16:03,314 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:16:03,413 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:16:03,423 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:16:03,426 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:16:03,433 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:16:03,436 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:16:03,439 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:16:03,444 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:16:03,449 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:16:03,453 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:16:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:16:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:09.793658+00:00 (in 5.605715 seconds) repositorygcworker stdout | 2023-08-08 06:16:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:04 UTC)" (scheduled at 2023-08-08 06:16:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:16:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:16:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:16:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:16:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:15.371682+00:00 (in 9.999475 seconds) notificationworker stdout | 2023-08-08 06:16:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:15 UTC)" (scheduled at 2023-08-08 06:16:05.371682+00:00) notificationworker stdout | 2023-08-08 06:16:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:16:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 5, 372598), True, datetime.datetime(2023, 8, 8, 6, 16, 5, 372598), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:16:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:16:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:16:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:16:07,179 [197] [DEBUG] [app] Starting request: urn:request:5c6d45ee-dda7-4742-b72a-f21663998731 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:16:07,179 [200] [DEBUG] [app] Starting request: urn:request:f1581dfb-bb44-4b7c-b008-316dda2cb231 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:16:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:16:07,185 [203] [DEBUG] [app] Starting request: urn:request:9b6133b3-ec0b-451f-80a6-de3e00d2af81 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:16:07,185 [203] [DEBUG] [app] Ending request: urn:request:9b6133b3-ec0b-451f-80a6-de3e00d2af81 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9b6133b3-ec0b-451f-80a6-de3e00d2af81', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:16:07,185 [209] [DEBUG] [app] Starting request: urn:request:ab0af307-dd0c-45d8-b030-c1a02d0c1b0b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:16:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:16:07,186 [209] [DEBUG] [app] Ending request: urn:request:ab0af307-dd0c-45d8-b030-c1a02d0c1b0b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ab0af307-dd0c-45d8-b030-c1a02d0c1b0b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:16:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:07,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:07,191 [197] [DEBUG] [app] Starting request: urn:request:17c820e8-6382-46f6-bf3b-0df84d15f0f2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:16:07,192 [197] [DEBUG] [app] Ending request: urn:request:17c820e8-6382-46f6-bf3b-0df84d15f0f2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:17c820e8-6382-46f6-bf3b-0df84d15f0f2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:07,192 [199] [DEBUG] [app] Starting request: urn:request:dd97fd6f-6732-4213-a611-63912fd0a863 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:16:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:07,193 [199] [DEBUG] [app] Ending request: urn:request:dd97fd6f-6732-4213-a611-63912fd0a863 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:dd97fd6f-6732-4213-a611-63912fd0a863', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:07,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:16:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:16:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:16:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:16:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:37.197412+00:00 (in 29.999488 seconds) buildlogsarchiver stdout | 2023-08-08 06:16:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:37 UTC)" (scheduled at 2023-08-08 06:16:07.197412+00:00) gunicorn-web stdout | 2023-08-08 06:16:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:16:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:16:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 16, 7, 198377), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:07,204 [197] [DEBUG] [app] Ending request: urn:request:5c6d45ee-dda7-4742-b72a-f21663998731 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5c6d45ee-dda7-4742-b72a-f21663998731', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:16:07,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:07,204 [200] [DEBUG] [app] Ending request: urn:request:f1581dfb-bb44-4b7c-b008-316dda2cb231 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f1581dfb-bb44-4b7c-b008-316dda2cb231', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:16:07,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:16:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:16:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:16:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:16:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:16:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:16:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:16:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:16:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:16:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:16:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:16:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:16:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:16:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:19.417660+00:00 (in 9.999430 seconds) gcworker stdout | 2023-08-08 06:16:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:19 UTC)" (scheduled at 2023-08-08 06:16:09.417660+00:00) gcworker stdout | 2023-08-08 06:16:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265769418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:16:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:16:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:16:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:16:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:16:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:04.187429+00:00 (in 54.393224 seconds) repositorygcworker stdout | 2023-08-08 06:16:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:09 UTC)" (scheduled at 2023-08-08 06:16:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:16:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:16:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 9, 794588), True, datetime.datetime(2023, 8, 8, 6, 16, 9, 794588), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:16:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:16:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:16:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:16:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:16:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:22.293757+00:00 (in 9.999460 seconds) chunkcleanupworker stdout | 2023-08-08 06:16:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:22 UTC)" (scheduled at 2023-08-08 06:16:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:16:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:16:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 12, 294891), True, datetime.datetime(2023, 8, 8, 6, 16, 12, 294891), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:16:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:16:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:16:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:16:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:16:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:31.283850+00:00 (in 18.185919 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:16:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:13 UTC)" (scheduled at 2023-08-08 06:16:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:16:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:16:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 13, 98403), True, datetime.datetime(2023, 8, 8, 6, 16, 13, 98403), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:16:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:16:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:16:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:13 UTC)" executed successfully builder stdout | 2023-08-08 06:16:13,324 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:16:13,333 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:16:13,334 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:16:13,338 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:16:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:16:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:25.371682+00:00 (in 9.999478 seconds) notificationworker stdout | 2023-08-08 06:16:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:25 UTC)" (scheduled at 2023-08-08 06:16:15.371682+00:00) notificationworker stdout | 2023-08-08 06:16:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:16:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 15, 372811), True, datetime.datetime(2023, 8, 8, 6, 16, 15, 372811), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:16:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:16:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:16:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:16:18,354 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:16:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:16:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:29.417660+00:00 (in 9.999437 seconds) gcworker stdout | 2023-08-08 06:16:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:29 UTC)" (scheduled at 2023-08-08 06:16:19.417660+00:00) gcworker stdout | 2023-08-08 06:16:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265779418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:16:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:16:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:16:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:29 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:16:19,667 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:16:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:16:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:26.890741+00:00 (in 7.202824 seconds) securityworker stdout | 2023-08-08 06:16:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:49 UTC)" (scheduled at 2023-08-08 06:16:19.687363+00:00) securityworker stdout | 2023-08-08 06:16:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:16:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:16:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:16:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:16:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:16:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:16:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 11, 19, 692221), 1, 2]) securityworker stdout | 2023-08-08 06:16:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 11, 19, 692221), 1, 2]) securityworker stdout | 2023-08-08 06:16:19,706 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:16:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:49 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:16:20,685 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:16:21,143 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:16:22,178 [197] [DEBUG] [app] Starting request: urn:request:322f93e9-6c87-4e4e-be0c-97cf90ba4b40 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:16:22,178 [199] [DEBUG] [app] Starting request: urn:request:797e355b-7839-4991-863f-cfcc99f807f5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:16:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:22,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:16:22,184 [208] [DEBUG] [app] Starting request: urn:request:b6f70034-c301-40f6-b4f5-a10d42deaa1f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:16:22,185 [206] [DEBUG] [app] Starting request: urn:request:a57584d3-1688-43d0-a361-03b10b04df7c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:16:22,185 [208] [DEBUG] [app] Ending request: urn:request:b6f70034-c301-40f6-b4f5-a10d42deaa1f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b6f70034-c301-40f6-b4f5-a10d42deaa1f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:16:22,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:16:22,185 [206] [DEBUG] [app] Ending request: urn:request:a57584d3-1688-43d0-a361-03b10b04df7c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a57584d3-1688-43d0-a361-03b10b04df7c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:22,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:16:22,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:22,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:22,190 [200] [DEBUG] [app] Starting request: urn:request:0d71d871-b1f5-407a-94e6-4aa35969eb7c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:16:22,191 [200] [DEBUG] [app] Ending request: urn:request:0d71d871-b1f5-407a-94e6-4aa35969eb7c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0d71d871-b1f5-407a-94e6-4aa35969eb7c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:22,191 [198] [DEBUG] [app] Starting request: urn:request:6b47c8f7-f4f0-478b-9cc1-af40a2e3f652 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:16:22,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:22,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:22,191 [198] [DEBUG] [app] Ending request: urn:request:6b47c8f7-f4f0-478b-9cc1-af40a2e3f652 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6b47c8f7-f4f0-478b-9cc1-af40a2e3f652', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:16:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:16:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:16:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:16:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:16:22,202 [199] [DEBUG] [app] Ending request: urn:request:797e355b-7839-4991-863f-cfcc99f807f5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:797e355b-7839-4991-863f-cfcc99f807f5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:16:22,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:22,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:16:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:16:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:16:22,203 [197] [DEBUG] [app] Ending request: urn:request:322f93e9-6c87-4e4e-be0c-97cf90ba4b40 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:322f93e9-6c87-4e4e-be0c-97cf90ba4b40', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:16:22,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:16:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:16:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:16:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:16:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:28.874906+00:00 (in 6.580563 seconds) chunkcleanupworker stdout | 2023-08-08 06:16:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:32 UTC)" (scheduled at 2023-08-08 06:16:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:16:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:16:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 22, 294898), True, datetime.datetime(2023, 8, 8, 6, 16, 22, 294898), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:16:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:16:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:16:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:32 UTC)" executed successfully builder stdout | 2023-08-08 06:16:23,348 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:16:23,357 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:16:23,357 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:16:23,362 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:16:23,866 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:16:25,131 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:16:25,134 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:16:25,136 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:16:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:16:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:35.371682+00:00 (in 9.999479 seconds) notificationworker stdout | 2023-08-08 06:16:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:35 UTC)" (scheduled at 2023-08-08 06:16:25.371682+00:00) notificationworker stdout | 2023-08-08 06:16:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:16:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 25, 372723), True, datetime.datetime(2023, 8, 8, 6, 16, 25, 372723), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:16:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:16:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:16:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:16:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:16:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:47.398551+00:00 (in 21.883702 seconds) exportactionlogsworker stdout | 2023-08-08 06:16:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:25 UTC)" (scheduled at 2023-08-08 06:16:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:16:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:16:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:16:25,677 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:16:25,982 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:16:26,169 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:16:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:16:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:49.687363+00:00 (in 22.796096 seconds) securityworker stdout | 2023-08-08 06:16:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:56 UTC)" (scheduled at 2023-08-08 06:16:26.890741+00:00) securityworker stdout | 2023-08-08 06:16:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:16:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:16:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:16:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:16:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:16:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:16:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:16:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:16:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:16:26,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:16:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 11, 26, 896601), 1, 2]) securityworker stdout | 2023-08-08 06:16:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 11, 26, 896601), 1, 2]) securityworker stdout | 2023-08-08 06:16:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:16:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:16:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:16:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:16:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:16:26,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:16:26,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:16:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:16:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:16:27,104 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:16:27,542 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:16:27,649 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:16:27,707 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:16:27,736 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:16:27,739 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} queuecleanupworker stdout | 2023-08-08 06:16:27,785 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} repositoryactioncounter stdout | 2023-08-08 06:16:27,808 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} chunkcleanupworker stdout | 2023-08-08 06:16:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:16:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:32.293757+00:00 (in 3.418186 seconds) chunkcleanupworker stdout | 2023-08-08 06:16:28,876 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:28 UTC)" (scheduled at 2023-08-08 06:16:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:16:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:16:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:28 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:16:29,138 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:16:29,298 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:16:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:16:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:39.417660+00:00 (in 9.999374 seconds) gcworker stdout | 2023-08-08 06:16:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:39 UTC)" (scheduled at 2023-08-08 06:16:29.417660+00:00) gcworker stdout | 2023-08-08 06:16:29,419 [61] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2023-08-08 06:16:29,426 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265789425, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:16:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:16:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:16:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:39 UTC)" executed successfully securityworker stdout | 2023-08-08 06:16:30,369 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:16:31,260 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} securityscanningnotificationworker stdout | 2023-08-08 06:16:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:16:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:54.284093+00:00 (in 22.999730 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:16:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:21:31 UTC)" (scheduled at 2023-08-08 06:16:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:16:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 16, 31, 284983), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:16:31,292 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 16, 31, 284983), True, datetime.datetime(2023, 8, 8, 6, 16, 31, 284983), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:16:31,294 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 16, 31, 284983), True, datetime.datetime(2023, 8, 8, 6, 16, 31, 284983), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 16, 31, 284983), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:16:31,295 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:16:31,296 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:21:31 UTC)" executed successfully servicekey stdout | 2023-08-08 06:16:31,353 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:16:31,821 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:16:32,257 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:16:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:16:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:42.293757+00:00 (in 9.999476 seconds) chunkcleanupworker stdout | 2023-08-08 06:16:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:42 UTC)" (scheduled at 2023-08-08 06:16:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:16:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:16:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 32, 294794), True, datetime.datetime(2023, 8, 8, 6, 16, 32, 294794), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:16:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:16:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:16:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:42 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:16:32,621 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-web stdout | 2023-08-08 06:16:32,676 [198] [DEBUG] [app] Starting request: urn:request:328686bb-b28d-4094-84e5-44b9d5230031 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:32,676 [197] [DEBUG] [app] Starting request: urn:request:ceb2f5fd-06b9-4c40-bfa5-1ee688c5243d (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:32,677 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:32,677 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:32,677 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:32,677 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:32,677 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:32,677 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:32,684 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:32,684 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:32,684 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:32,684 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:32,684 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:32,684 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:32,684 [198] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:16:32,684 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:16:32,684 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:32,684 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:32,685 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:32,685 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:32,685 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:32,685 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:32,685 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:32,685 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:32,685 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 28, 32, 685264), datetime.datetime(2023, 8, 8, 6, 16, 32, 685324)]) gunicorn-web stdout | 2023-08-08 06:16:32,686 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:16:32,687 [197] [DEBUG] [app] Ending request: urn:request:ceb2f5fd-06b9-4c40-bfa5-1ee688c5243d (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:ceb2f5fd-06b9-4c40-bfa5-1ee688c5243d', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:32,688 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:32 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 62 0.013) gunicorn-web stdout | 2023-08-08 06:16:32,688 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:32 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:32,691 [198] [DEBUG] [app] Ending request: urn:request:328686bb-b28d-4094-84e5-44b9d5230031 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:328686bb-b28d-4094-84e5-44b9d5230031', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:32,692 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:32 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 870 0.017) gunicorn-web stdout | 2023-08-08 06:16:32,692 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:32 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/github/build/d5b40e12-f92c-42e5-9c25-9e76946bc5a3" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" expiredappspecifictokenworker stdout | 2023-08-08 06:16:32,961 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:16:33,371 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:16:33,380 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:16:33,380 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:16:33,384 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:16:33,423 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:16:33,432 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:16:33,435 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:16:33,443 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:16:33,446 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:16:33,450 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:16:33,453 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:16:33,457 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:16:33,463 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-web stdout | 2023-08-08 06:16:34,416 [197] [DEBUG] [app] Starting request: urn:request:701d0929-d1cd-46b1-adfa-94ccb97df3b0 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:34,416 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,416 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,417 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:34,419 [200] [DEBUG] [app] Starting request: urn:request:61f66fed-c628-4052-a0e0-e217d7c989bd (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:34,419 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,419 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,419 [198] [DEBUG] [app] Starting request: urn:request:e17044c1-0502-412c-855a-fb26a6e2b0f6 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:34,419 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,419 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,420 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:34,420 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:34,423 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:34,423 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:34,423 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,424 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 06:16:34,425 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:34,426 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:34,426 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,426 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:34,426 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:34,426 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,426 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,426 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:34,426 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:34,426 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:34,426 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:34,426 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:34,426 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:34,427 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:34,427 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:34,427 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:34,427 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:34,427 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:34,428 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 06:16:34,428 [200] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'whuquay', 101]) gunicorn-web stdout | 2023-08-08 06:16:34,429 [198] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE ((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) ORDER BY rid LIMIT %s', [3, 1, 'build_test', 101]) gunicorn-web stdout | 2023-08-08 06:16:34,430 [197] [DEBUG] [app] Ending request: urn:request:701d0929-d1cd-46b1-adfa-94ccb97df3b0 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:701d0929-d1cd-46b1-adfa-94ccb97df3b0', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:34,430 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:34 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 501 0.016) gunicorn-web stdout | 2023-08-08 06:16:34,431 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:34 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:34,432 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 06:16:34,432 [198] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Max("t1"."lifetime_start_ms") FROM "tag" AS "t1" WHERE ((("t1"."repository_id" IN (%s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) GROUP BY "t1"."repository_id"', [2, None, 1691475394431, False]) gunicorn-web stdout | 2023-08-08 06:16:34,433 [198] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Sum("t1"."count") FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" IN (%s)) AND ("t1"."date" >= %s)) GROUP BY "t1"."repository_id"', [2, datetime.date(2023, 8, 1)]) gunicorn-web stdout | 2023-08-08 06:16:34,434 [200] [DEBUG] [app] Ending request: urn:request:61f66fed-c628-4052-a0e0-e217d7c989bd (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:61f66fed-c628-4052-a0e0-e217d7c989bd', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'whuquay', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:34,435 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:34,435 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:34 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:34,435 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:34 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 103 0.017) gunicorn-web stdout | 2023-08-08 06:16:34,437 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:34,439 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 06:16:34,440 [198] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:34,442 [198] [DEBUG] [app] Ending request: urn:request:e17044c1-0502-412c-855a-fb26a6e2b0f6 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:e17044c1-0502-412c-855a-fb26a6e2b0f6', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'false', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:34,442 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:34 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/2.0" 200 288 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.025 114 0.024) gunicorn-web stdout | 2023-08-08 06:16:34,443 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:34 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/1.0" 200 288 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:16:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:16:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:45.371682+00:00 (in 9.999482 seconds) notificationworker stdout | 2023-08-08 06:16:35,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:45 UTC)" (scheduled at 2023-08-08 06:16:35.371682+00:00) notificationworker stdout | 2023-08-08 06:16:35,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:16:35,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 35, 373256), True, datetime.datetime(2023, 8, 8, 6, 16, 35, 373256), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:16:35,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:16:35,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:16:35,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:16:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:16:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:54.393627+00:00 (in 17.813282 seconds) namespacegcworker stdout | 2023-08-08 06:16:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:36 UTC)" (scheduled at 2023-08-08 06:16:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:16:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:16:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:16:37,178 [197] [DEBUG] [app] Starting request: urn:request:59690035-86b6-4c0f-b73a-2a323e0b0815 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:16:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:37,180 [198] [DEBUG] [app] Starting request: urn:request:cb8056aa-5e85-4cfc-815f-c5ea6cdb988b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:16:37,182 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:16:37,184 [207] [DEBUG] [app] Starting request: urn:request:2be5a12f-6b94-4ca6-a199-a8e99e3092bf (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:16:37,184 [207] [DEBUG] [app] Ending request: urn:request:2be5a12f-6b94-4ca6-a199-a8e99e3092bf (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2be5a12f-6b94-4ca6-a199-a8e99e3092bf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:37,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:16:37,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:16:37,185 [204] [DEBUG] [app] Starting request: urn:request:1df7e982-093b-4de8-95f8-27e30767bee2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:16:37,186 [204] [DEBUG] [app] Ending request: urn:request:1df7e982-093b-4de8-95f8-27e30767bee2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1df7e982-093b-4de8-95f8-27e30767bee2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:37,186 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:16:37,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:37,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:37,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:37,188 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:37,190 [200] [DEBUG] [app] Starting request: urn:request:5de39ffe-8252-4049-bf08-682653decc03 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:16:37,190 [200] [DEBUG] [app] Ending request: urn:request:5de39ffe-8252-4049-bf08-682653decc03 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5de39ffe-8252-4049-bf08-682653decc03', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:37,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:16:37,190 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:37,191 [197] [DEBUG] [app] Starting request: urn:request:3cf1637f-086b-48f8-9aae-e439600ef14c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:16:37,191 [197] [DEBUG] [app] Ending request: urn:request:3cf1637f-086b-48f8-9aae-e439600ef14c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3cf1637f-086b-48f8-9aae-e439600ef14c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:16:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:37,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:37,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:16:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:16:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:16:37,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:16:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:16:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:07.197412+00:00 (in 29.999362 seconds) buildlogsarchiver stdout | 2023-08-08 06:16:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:07 UTC)" (scheduled at 2023-08-08 06:16:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:16:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 16, 37, 198484), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,203 [197] [DEBUG] [app] Ending request: urn:request:59690035-86b6-4c0f-b73a-2a323e0b0815 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:59690035-86b6-4c0f-b73a-2a323e0b0815', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:16:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:37,203 [198] [DEBUG] [app] Ending request: urn:request:cb8056aa-5e85-4cfc-815f-c5ea6cdb988b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cb8056aa-5e85-4cfc-815f-c5ea6cdb988b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:16:37,203 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:16:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:16:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:16:37,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:16:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:16:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:16:37,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:16:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:16:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:16:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:16:37,457 [199] [DEBUG] [app] Starting request: urn:request:ecd4b1b9-5ca1-46af-9048-5ee1b09a65fc (/api/v1/organization/build_test) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:37,457 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,457 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,458 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,464 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:37,464 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:37,464 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,465 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,466 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,466 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,466 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,466 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,467 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:16:37,470 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,470 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,470 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,470 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:37,471 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t2" ON ("t1"."role_id" = "t2"."id") WHERE ("t1"."organization_id" = %s)', [2]) gunicorn-web stdout | 2023-08-08 06:16:37,472 [199] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "repositorypermission" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [1]) gunicorn-web stdout | 2023-08-08 06:16:37,473 [199] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "teammember" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [1]) gunicorn-web stdout | 2023-08-08 06:16:37,474 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,474 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,474 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,474 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 06:16:37,476 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,478 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691475397477, 0, 2, None]) gunicorn-web stdout | 2023-08-08 06:16:37,481 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,482 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 06:16:37,484 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,485 [199] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691475397485, 0, 2]) gunicorn-web stdout | 2023-08-08 06:16:37,487 [199] [DEBUG] [app] Ending request: urn:request:ecd4b1b9-5ca1-46af-9048-5ee1b09a65fc (/api/v1/organization/build_test) {'endpoint': 'api.organization', 'request_id': 'urn:request:ecd4b1b9-5ca1-46af-9048-5ee1b09a65fc', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test', 'path': '/api/v1/organization/build_test', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:37,488 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:37,488 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:37 +0000] "GET /api/v1/organization/build_test HTTP/1.0" 200 778 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /api/v1/organization/build_test HTTP/2.0" 200 778 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.033 483 0.033) gunicorn-web stdout | 2023-08-08 06:16:37,955 [198] [DEBUG] [app] Starting request: urn:request:720b9372-15f0-4270-beeb-367ce7ebee7c (/api/v1/organization/build_test/quota) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:37,955 [200] [DEBUG] [app] Starting request: urn:request:fe571db0-9720-451f-b6dc-a39edc566f3f (/api/v1/organization/build_test/proxycache) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:37,955 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,955 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,955 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,955 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,956 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,956 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,963 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:37,963 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:37,963 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:37,963 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,963 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:37,963 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,963 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,963 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,963 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,963 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,963 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:37,963 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,963 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,963 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,964 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:16:37,965 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:16:37,967 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,967 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,967 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,967 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,967 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,967 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:16:37,967 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:37,967 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:37,968 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', True, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,968 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:37,969 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 06:16:37,970 [200] [DEBUG] [app] Ending request: urn:request:fe571db0-9720-451f-b6dc-a39edc566f3f (/api/v1/organization/build_test/proxycache) {'endpoint': 'api.organizationproxycacheconfig', 'request_id': 'urn:request:fe571db0-9720-451f-b6dc-a39edc566f3f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/proxycache', 'path': '/api/v1/organization/build_test/proxycache', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:37,971 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:37,971 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."quota_id", "t1"."quota_type_id", "t1"."percent_of_limit" FROM "quotalimits" AS "t1" INNER JOIN "quotatype" AS "t2" ON ("t1"."quota_type_id" = "t2"."id") WHERE ("t1"."quota_id" = %s)', [None]) gunicorn-web stdout | 2023-08-08 06:16:37,971 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:37 +0000] "GET /api/v1/organization/build_test/proxycache HTTP/1.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /api/v1/organization/build_test/proxycache HTTP/2.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 66 0.018) gunicorn-web stdout | 2023-08-08 06:16:37,972 [198] [DEBUG] [app] Ending request: urn:request:720b9372-15f0-4270-beeb-367ce7ebee7c (/api/v1/organization/build_test/quota) {'endpoint': 'api.organizationquotalist', 'request_id': 'urn:request:720b9372-15f0-4270-beeb-367ce7ebee7c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/quota', 'path': '/api/v1/organization/build_test/quota', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:37,973 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:37 +0000] "GET /api/v1/organization/build_test/quota HTTP/2.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 407 0.021) gunicorn-web stdout | 2023-08-08 06:16:37,973 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:37 +0000] "GET /api/v1/organization/build_test/quota HTTP/1.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:38 +0000] "GET /static/img/creditcards/credit.png HTTP/2.0" 200 1776 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 92 -) gunicorn-web stdout | 2023-08-08 06:16:38,094 [197] [DEBUG] [app] Starting request: urn:request:131f4ba4-d542-45dd-90a4-a7c8aef55b87 (/api/v1/organization/build_test/prototypes) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:38,095 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:38,095 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:38,095 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:38,102 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:38,102 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:38,102 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:38,102 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:38,102 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,102 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,102 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,103 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:16:38,106 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,106 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,106 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,106 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:38,107 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:38,108 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" LEFT OUTER JOIN "user" AS "t2" ON ("t2"."id" = "t1"."activating_user_id") LEFT OUTER JOIN "user" AS "t3" ON ("t3"."id" = "t1"."delegate_user_id") LEFT OUTER JOIN "team" AS "t4" ON ("t4"."id" = "t1"."delegate_team_id") LEFT OUTER JOIN "role" AS "t5" ON ("t5"."id" = "t1"."role_id") WHERE ("t1"."org_id" = %s)', [2]) gunicorn-web stdout | 2023-08-08 06:16:38,110 [197] [DEBUG] [app] Ending request: urn:request:131f4ba4-d542-45dd-90a4-a7c8aef55b87 (/api/v1/organization/build_test/prototypes) {'endpoint': 'api.permissionprototypelist', 'request_id': 'urn:request:131f4ba4-d542-45dd-90a4-a7c8aef55b87', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/prototypes', 'path': '/api/v1/organization/build_test/prototypes', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:38,111 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:38 +0000] "GET /api/v1/organization/build_test/prototypes HTTP/2.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 66 0.019) gunicorn-web stdout | 2023-08-08 06:16:38,111 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:38 +0000] "GET /api/v1/organization/build_test/prototypes HTTP/1.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:38,116 [197] [DEBUG] [app] Starting request: urn:request:2bd26253-ab0a-4b51-a01a-e0457335c959 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:38,116 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:38,116 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:38,117 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:38,122 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:38,122 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:38,122 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:38,122 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:38,123 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,123 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,123 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:38,123 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:38,123 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:38,125 [197] [DEBUG] [peewee] ('(SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t1"."visibility_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") WHERE ((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'build_test', 1, 3, 1, 'build_test', 101]) gunicorn-web stdout | 2023-08-08 06:16:38,129 [197] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Max("t1"."lifetime_start_ms") FROM "tag" AS "t1" WHERE ((("t1"."repository_id" IN (%s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) GROUP BY "t1"."repository_id"', [2, None, 1691475398128, False]) gunicorn-web stdout | 2023-08-08 06:16:38,130 [197] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Sum("t1"."count") FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" IN (%s)) AND ("t1"."date" >= %s)) GROUP BY "t1"."repository_id"', [2, datetime.date(2023, 8, 1)]) gunicorn-web stdout | 2023-08-08 06:16:38,132 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2023-08-08 06:16:38,135 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:38,136 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2023-08-08 06:16:38,137 [197] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:38,139 [197] [DEBUG] [app] Ending request: urn:request:2bd26253-ab0a-4b51-a01a-e0457335c959 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:2bd26253-ab0a-4b51-a01a-e0457335c959', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'true', 'quota': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:38,139 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:38 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true HTTP/2.0" 200 288 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.024 106 0.025) gunicorn-web stdout | 2023-08-08 06:16:38,140 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:38 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true HTTP/1.0" 200 288 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:39 +0000] "GET /static/fonts/glyphicons-halflings-regular.woff2 HTTP/2.0" 200 18028 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/static/ldn/bootstrap.min.css-13f668494557.css" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 572 -) gcworker stdout | 2023-08-08 06:16:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:16:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:49.417660+00:00 (in 9.999418 seconds) gcworker stdout | 2023-08-08 06:16:39,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:49 UTC)" (scheduled at 2023-08-08 06:16:39.417660+00:00) gcworker stdout | 2023-08-08 06:16:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265799419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:16:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:16:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:16:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:16:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:16:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:52.293757+00:00 (in 9.999512 seconds) chunkcleanupworker stdout | 2023-08-08 06:16:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:52 UTC)" (scheduled at 2023-08-08 06:16:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:16:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:16:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 42, 294776), True, datetime.datetime(2023, 8, 8, 6, 16, 42, 294776), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:16:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:16:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:16:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:52 UTC)" executed successfully builder stdout | 2023-08-08 06:16:43,395 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:16:43,403 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:16:43,403 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:16:43,407 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:16:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:16:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:55.371682+00:00 (in 9.999490 seconds) notificationworker stdout | 2023-08-08 06:16:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:55 UTC)" (scheduled at 2023-08-08 06:16:45.371682+00:00) notificationworker stdout | 2023-08-08 06:16:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:16:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 45, 372727), True, datetime.datetime(2023, 8, 8, 6, 16, 45, 372727), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:16:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:16:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:16:45,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:55 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:16:46,695 [198] [DEBUG] [app] Starting request: urn:request:05d33fe6-cbd4-47db-8889-e46180922907 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:46,696 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:46,696 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:46,696 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,702 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:46,703 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:46,703 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:46,704 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:46,704 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:46,704 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:46,704 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:46,706 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:16:46,708 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:46,708 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:46,708 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:16:46,708 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:46,709 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,712 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,713 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,715 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1"', []) gunicorn-web stdout | 2023-08-08 06:16:46,716 [198] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [2, 'custom', 2, '', '701a93e9-baf8-4db2-9544-e3d916850a71', 1, False, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,718 [198] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [3, 0, datetime.date(2023, 8, 7)]) gunicorn-web stdout | 2023-08-08 06:16:46,719 [198] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [3, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,720 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,720 [198] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [1, 3, 1]) gunicorn-web stdout | 2023-08-08 06:16:46,724 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_user_id" IS %s))', [2, 1, None, None]) gunicorn-web stdout | 2023-08-08 06:16:46,726 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_team_id" IS %s))', [2, 1, None, None]) gunicorn-web stdout | 2023-08-08 06:16:46,727 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,729 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:46,730 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "logentrykind" AS "t1"', []) gunicorn-web stdout | 2023-08-08 06:16:46,732 [198] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [14, 2, 1, 3, datetime.datetime(2023, 8, 8, 6, 16, 46, 728817), '10.130.2.1', '{"repo": "custom", "namespace": "build_test"}']) gunicorn-web stdout | 2023-08-08 06:16:46,735 [198] [DEBUG] [app] Ending request: urn:request:05d33fe6-cbd4-47db-8889-e46180922907 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:05d33fe6-cbd4-47db-8889-e46180922907', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository', 'path': '/api/v1/repository', 'parameters': {}, 'json_body': {'namespace': 'build_test', 'repository': 'custom', 'visibility': 'private', 'description': '', 'repo_kind': 'image'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:46,736 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:46,736 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:46 +0000] "POST /api/v1/repository HTTP/1.0" 201 63 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:46 +0000] "POST /api/v1/repository HTTP/2.0" 201 63 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.042 1038 0.042) gunicorn-web stdout | 2023-08-08 06:16:47,110 [197] [DEBUG] [app] Starting request: urn:request:648fb57e-bbd0-4372-8563-e57a66e7761d (/api/v1/repository/build_test/custom/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:47,111 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,111 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,111 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,111 [199] [DEBUG] [app] Starting request: urn:request:67fbd7ef-f430-42c0-8bc9-6a87e6ebb902 (/api/v1/repository/build_test/custom) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:47,112 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,112 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,112 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,118 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:47,118 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:47,118 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,118 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:47,118 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:47,118 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,118 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:47,118 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,118 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,119 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,119 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,119 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:47,119 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,119 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,119 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,119 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,121 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:16:47,121 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:16:47,127 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,127 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,127 [199] [DEBUG] [endpoints.api.repository] Get repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:47,128 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,128 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,129 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [3, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,131 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,131 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [3, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:16:47,132 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,132 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,133 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [3, None, 1691475407132, False, 101, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,133 [199] [DEBUG] [app] Ending request: urn:request:67fbd7ef-f430-42c0-8bc9-6a87e6ebb902 (/api/v1/repository/build_test/custom) {'endpoint': 'api.repository', 'request_id': 'urn:request:67fbd7ef-f430-42c0-8bc9-6a87e6ebb902', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/custom', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:47,134 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:47 +0000] "GET /api/v1/repository/build_test/custom?includeStats=true&includeTags=false HTTP/2.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.024 87 0.025) gunicorn-web stdout | 2023-08-08 06:16:47,134 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:47 +0000] "GET /api/v1/repository/build_test/custom?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:47,135 [197] [DEBUG] [app] Ending request: urn:request:648fb57e-bbd0-4372-8563-e57a66e7761d (/api/v1/repository/build_test/custom/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:648fb57e-bbd0-4372-8563-e57a66e7761d', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/custom/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:47,136 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:47,136 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:47 +0000] "GET /api/v1/repository/build_test/custom/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:47 +0000] "GET /api/v1/repository/build_test/custom/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 536 0.027) exportactionlogsworker stdout | 2023-08-08 06:16:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:16:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:25.514341+00:00 (in 38.115265 seconds) exportactionlogsworker stdout | 2023-08-08 06:16:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:47 UTC)" (scheduled at 2023-08-08 06:16:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:16:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:16:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 47, 399594), True, datetime.datetime(2023, 8, 8, 6, 16, 47, 399594), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:16:47,406 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:16:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:16:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:47 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:16:47,719 [197] [DEBUG] [app] Starting request: urn:request:79cb2877-04c9-40d8-a4c0-6f0bb621b7d9 (/repository/build_test/custom/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:47,720 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,728 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,729 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,732 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['custom', 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,738 [197] [DEBUG] [app] Ending request: urn:request:79cb2877-04c9-40d8-a4c0-6f0bb621b7d9 (/repository/build_test/custom/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:79cb2877-04c9-40d8-a4c0-6f0bb621b7d9', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/status?token=701a93e9-baf8-4db2-9544-e3d916850a71', 'path': '/repository/build_test/custom/status', 'parameters': {'token': '701a93e9-baf8-4db2-9544-e3d916850a71'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:47,739 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:47,739 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:47 +0000] "GET /repository/build_test/custom/status?token=701a93e9-baf8-4db2-9544-e3d916850a71 HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:47 +0000] "GET /repository/build_test/custom/status?token=701a93e9-baf8-4db2-9544-e3d916850a71 HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.022 469 0.022) gunicorn-web stdout | 2023-08-08 06:16:47,860 [197] [DEBUG] [app] Starting request: urn:request:19ea6007-cee1-46e6-aed2-04398c1dc8d1 (/api/v1/repository/build_test/custom/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:47,860 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,860 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,861 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,867 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:47,867 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:47,867 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,867 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:47,867 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:47,867 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,867 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,867 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,870 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:16:47,875 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:16:47,877 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:47,880 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['custom', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 06:16:47,884 [197] [DEBUG] [app] Ending request: urn:request:19ea6007-cee1-46e6-aed2-04398c1dc8d1 (/api/v1/repository/build_test/custom/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:19ea6007-cee1-46e6-aed2-04398c1dc8d1', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/build/?limit=3', 'path': '/api/v1/repository/build_test/custom/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:47,885 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:47,885 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:47 +0000] "GET /api/v1/repository/build_test/custom/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:47 +0000] "GET /api/v1/repository/build_test/custom/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?namespace=build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 73 0.026) notificationworker stdout | 2023-08-08 06:16:48,391 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:16:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:16:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:59.417660+00:00 (in 9.999443 seconds) gcworker stdout | 2023-08-08 06:16:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:59 UTC)" (scheduled at 2023-08-08 06:16:49.417660+00:00) gcworker stdout | 2023-08-08 06:16:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265809418, 1209600, True, 3, 500, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:49,423 [200] [DEBUG] [app] Starting request: urn:request:55f930b1-580e-4136-8f6d-b6914e8d49ef (/api/v1/repository/build_test/custom/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:49,423 [197] [DEBUG] [app] Starting request: urn:request:4446671d-0ace-4ebb-a37e-b9812d7e770c (/api/v1/repository/build_test/custom/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:49,423 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:49,423 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:49,423 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:49,423 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:49,424 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:49,424 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gcworker stdout | 2023-08-08 06:16:49,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:16:49,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:16:49,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:16:59 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:16:49,429 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:49,429 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:49,429 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:49,430 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:49,430 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:49,430 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:49,430 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:49,430 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:49,431 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:49,431 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:49,431 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:49,431 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:49,431 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:49,431 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:49,431 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:49,431 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:49,432 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:16:49,433 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:16:49,438 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:16:49,439 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:49,439 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:16:49,441 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:49,442 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'custom']) gunicorn-web stdout | 2023-08-08 06:16:49,444 [200] [DEBUG] [app] Ending request: urn:request:55f930b1-580e-4136-8f6d-b6914e8d49ef (/api/v1/repository/build_test/custom/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:55f930b1-580e-4136-8f6d-b6914e8d49ef', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/', 'path': '/api/v1/repository/build_test/custom/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:49,445 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['custom', 'build_test', 10]) gunicorn-web stdout | 2023-08-08 06:16:49,445 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:49,445 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:49 +0000] "GET /api/v1/repository/build_test/custom/trigger/ HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:49 +0000] "GET /api/v1/repository/build_test/custom/trigger/ HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.024 160 0.025) gunicorn-web stdout | 2023-08-08 06:16:49,449 [197] [DEBUG] [app] Ending request: urn:request:4446671d-0ace-4ebb-a37e-b9812d7e770c (/api/v1/repository/build_test/custom/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:4446671d-0ace-4ebb-a37e-b9812d7e770c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/build/?limit=10', 'path': '/api/v1/repository/build_test/custom/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:49,449 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:49,450 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:49 +0000] "GET /api/v1/repository/build_test/custom/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:49 +0000] "GET /api/v1/repository/build_test/custom/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 73 0.028) logrotateworker stdout | 2023-08-08 06:16:49,685 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} securityworker stdout | 2023-08-08 06:16:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:16:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:16:56.890741+00:00 (in 7.202877 seconds) securityworker stdout | 2023-08-08 06:16:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:19 UTC)" (scheduled at 2023-08-08 06:16:49.687363+00:00) securityworker stdout | 2023-08-08 06:16:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:16:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:16:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:16:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:16:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:16:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:19 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:16:50,713 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:16:51,158 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:16:51,642 [197] [DEBUG] [app] Starting request: urn:request:1d3865f0-32a5-4e76-bfc0-236052ed0450 (/customtrigger/setup/build_test/custom) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:51,643 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:51,643 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:51,643 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:51,650 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:51,650 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:51,650 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:51,650 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:51,650 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:51,650 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:51,653 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:16:51,659 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:16:51,660 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:51,662 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:51,663 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "buildtriggerservice" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['custom-git', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:51,664 [197] [DEBUG] [peewee] ('INSERT INTO "repositorybuildtrigger" ("uuid", "service_id", "repository_id", "connected_user_id", "secure_auth_token", "fully_migrated", "config", "pull_robot_id", "enabled", "disabled_datetime", "successive_failure_count", "successive_internal_error_count") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repositorybuildtrigger"."id"', ['1f727fd3-07cc-4975-bada-b401b1746237', 4, 3, 1, None, False, '{}', None, True, datetime.datetime(2023, 8, 8, 6, 16, 51, 664145), 0, 0]) gunicorn-web stdout | 2023-08-08 06:16:51,669 [197] [DEBUG] [endpoints.web] Redirecting to full url: /repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237 gunicorn-web stdout | 2023-08-08 06:16:51,670 [197] [DEBUG] [app] Ending request: urn:request:1d3865f0-32a5-4e76-bfc0-236052ed0450 (/customtrigger/setup/build_test/custom) {'endpoint': 'web.attach_custom_build_trigger', 'request_id': 'urn:request:1d3865f0-32a5-4e76-bfc0-236052ed0450', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/customtrigger/setup/build_test/custom', 'path': '/customtrigger/setup/build_test/custom', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:51,671 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:51 +0000] "GET /customtrigger/setup/build_test/custom HTTP/2.0" 302 355 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 524 0.030) gunicorn-web stdout | 2023-08-08 06:16:51,671 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:51 +0000] "GET /customtrigger/setup/build_test/custom HTTP/1.0" 302 355 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:52,028 [198] [DEBUG] [app] Starting request: urn:request:a41be04c-3f91-43af-993a-31eb6033dbd1 (/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:52,030 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,032 [198] [DEBUG] [app] Ending request: urn:request:a41be04c-3f91-43af-993a-31eb6033dbd1 (/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237) {'endpoint': 'web.buildtrigger', 'request_id': 'urn:request:a41be04c-3f91-43af-993a-31eb6033dbd1', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237', 'path': '/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:52,033 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:52 +0000] "GET /repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237 HTTP/1.0" 200 85192 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237 HTTP/2.0" 200 13315 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.007 433 0.006) gunicorn-web stdout | 2023-08-08 06:16:52,179 [197] [DEBUG] [app] Starting request: urn:request:2720f48e-db92-4092-ad49-2b17683c0325 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:16:52,179 [200] [DEBUG] [app] Starting request: urn:request:11b0a00f-4387-428d-8440-73f5b323046c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:16:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:16:52,185 [210] [DEBUG] [app] Starting request: urn:request:2968535d-aa7d-4d3d-88cd-bb287e2fd2da (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:16:52,185 [204] [DEBUG] [app] Starting request: urn:request:937c19da-e905-4f30-8cca-2b791af5983a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:16:52,185 [210] [DEBUG] [app] Ending request: urn:request:2968535d-aa7d-4d3d-88cd-bb287e2fd2da (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2968535d-aa7d-4d3d-88cd-bb287e2fd2da', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:16:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:16:52,186 [204] [DEBUG] [app] Ending request: urn:request:937c19da-e905-4f30-8cca-2b791af5983a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:937c19da-e905-4f30-8cca-2b791af5983a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:16:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:16:52,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:16:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:16:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:16:52,191 [197] [DEBUG] [app] Starting request: urn:request:6f1eff4f-4059-4c2b-8d50-e2172d47fa1d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:16:52,191 [199] [DEBUG] [app] Starting request: urn:request:29286ff8-1d9c-49fb-8b1f-d5609e03cc07 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:16:52,191 [197] [DEBUG] [app] Ending request: urn:request:6f1eff4f-4059-4c2b-8d50-e2172d47fa1d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6f1eff4f-4059-4c2b-8d50-e2172d47fa1d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:16:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:52,192 [199] [DEBUG] [app] Ending request: urn:request:29286ff8-1d9c-49fb-8b1f-d5609e03cc07 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:29286ff8-1d9c-49fb-8b1f-d5609e03cc07', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:16:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:16:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:16:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:16:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:16:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:16:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:16:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:16:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:16:52,202 [197] [DEBUG] [app] Ending request: urn:request:2720f48e-db92-4092-ad49-2b17683c0325 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2720f48e-db92-4092-ad49-2b17683c0325', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:16:52,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:16:52,203 [200] [DEBUG] [app] Ending request: urn:request:11b0a00f-4387-428d-8440-73f5b323046c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:11b0a00f-4387-428d-8440-73f5b323046c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:16:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:16:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:16:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:16:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:16:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:16:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:02.293757+00:00 (in 9.999503 seconds) chunkcleanupworker stdout | 2023-08-08 06:16:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:02 UTC)" (scheduled at 2023-08-08 06:16:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:16:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:16:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 52, 294758), True, datetime.datetime(2023, 8, 8, 6, 16, 52, 294758), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:16:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:16:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:16:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:02 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:16:52,923 [197] [DEBUG] [app] Starting request: urn:request:7f27aa61-5852-4e3d-9271-434be3985aec (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:52,924 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,924 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,924 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:52,931 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:52,931 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:52,932 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,932 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,932 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:52,932 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:52,932 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:52,933 [197] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2023-08-08 06:16:52,936 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,936 [197] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2023-08-08 06:16:52,938 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2023-08-08 06:16:52,939 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:52,940 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 06:16:52,942 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:52,943 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691475412942, 0, 1, None]) gunicorn-web stdout | 2023-08-08 06:16:52,946 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:52,947 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2023-08-08 06:16:52,949 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:52,949 [200] [DEBUG] [app] Starting request: urn:request:15b3ada5-d7af-4937-9f15-d816fc973e8e (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:52,949 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,949 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,950 [197] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691475412949, 0, 1]) gunicorn-web stdout | 2023-08-08 06:16:52,950 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:52,951 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,951 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,953 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:16:52,954 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:52,954 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:16:52,955 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:16:52,955 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:16:52,955 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,955 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,955 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,955 [197] [DEBUG] [app] Ending request: urn:request:7f27aa61-5852-4e3d-9271-434be3985aec (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:7f27aa61-5852-4e3d-9271-434be3985aec', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:52,956 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:52,956 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:52,956 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:52,956 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:16:52,957 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 06:16:52,957 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:52 +0000] "GET /api/v1/user/ HTTP/1.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /api/v1/user/ HTTP/2.0" 200 934 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.034 847 0.034) gunicorn-web stdout | 2023-08-08 06:16:52,958 [200] [DEBUG] [app] Ending request: urn:request:15b3ada5-d7af-4937-9f15-d816fc973e8e (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:15b3ada5-d7af-4937-9f15-d816fc973e8e', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:52,959 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:52 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.011 47 0.012) gunicorn-web stdout | 2023-08-08 06:16:52,959 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:52 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:53,031 [199] [DEBUG] [app] Starting request: urn:request:fd470a80-7c95-4657-8bd5-1f3e59f78e10 (/api/v1/repository/build_test/custom) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:53,031 [197] [DEBUG] [app] Starting request: urn:request:98f75868-7fe0-4eec-8046-6b7acaa777bd (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:53,031 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,031 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,031 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,031 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,032 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:53,032 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:53,038 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:53,038 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:53,038 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:53,038 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,038 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:53,038 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,038 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:53,038 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,038 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,038 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,038 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,038 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:53,038 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,039 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,039 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,039 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,041 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:16:53,041 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:16:53,047 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,047 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,047 [199] [DEBUG] [endpoints.api.repository] Get repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:16:53,048 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:53,048 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:53,049 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [3, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:53,051 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [3, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:16:53,051 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:53,052 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,052 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,053 [199] [DEBUG] [app] Ending request: urn:request:fd470a80-7c95-4657-8bd5-1f3e59f78e10 (/api/v1/repository/build_test/custom) {'endpoint': 'api.repository', 'request_id': 'urn:request:fd470a80-7c95-4657-8bd5-1f3e59f78e10', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom?includeTags=false', 'path': '/api/v1/repository/build_test/custom', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:53,053 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:53 +0000] "GET /api/v1/repository/build_test/custom?includeTags=false HTTP/2.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.025 75 0.024) gunicorn-web stdout | 2023-08-08 06:16:53,054 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:53 +0000] "GET /api/v1/repository/build_test/custom?includeTags=false HTTP/1.0" 200 339 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:53,054 [197] [DEBUG] [app] Ending request: urn:request:98f75868-7fe0-4eec-8046-6b7acaa777bd (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:98f75868-7fe0-4eec-8046-6b7acaa777bd', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:53,055 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:53 +0000] "GET /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237 HTTP/2.0" 200 214 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 93 0.026) gunicorn-web stdout | 2023-08-08 06:16:53,055 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:53 +0000] "GET /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237 HTTP/1.0" 200 214 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:53,376 [200] [DEBUG] [app] Starting request: urn:request:2ae97ea9-8407-4b84-93da-c6f5ce0913ad (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:53,376 [197] [DEBUG] [app] Starting request: urn:request:94f93e7c-074f-4255-b576-345c108bfa29 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:16:53,376 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,376 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,376 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,376 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,377 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:53,377 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:16:53,383 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:53,383 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:53,383 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,383 [197] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:16:53,383 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,383 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,383 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,383 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,383 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:16:53,384 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:16:53,384 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,384 [200] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2023-08-08 06:16:53,384 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:16:53,384 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,384 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,384 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:16:53,384 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2023, 8, 11, 1, 28, 53, 384061), datetime.datetime(2023, 8, 8, 6, 16, 53, 384112)]) gunicorn-web stdout | 2023-08-08 06:16:53,386 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:16:53,386 [197] [DEBUG] [app] Ending request: urn:request:94f93e7c-074f-4255-b576-345c108bfa29 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:94f93e7c-074f-4255-b576-345c108bfa29', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:53,386 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:53 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 62 0.012) gunicorn-web stdout | 2023-08-08 06:16:53,387 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:53 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:16:53,390 [200] [DEBUG] [app] Ending request: urn:request:2ae97ea9-8407-4b84-93da-c6f5ce0913ad (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:2ae97ea9-8407-4b84-93da-c6f5ce0913ad', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:16:53,391 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:16:53 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 397 0.016) gunicorn-web stdout | 2023-08-08 06:16:53,391 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:16:53 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" builder stdout | 2023-08-08 06:16:53,417 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:16:53,425 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:16:53,425 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:16:53,429 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:16:53,881 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:16:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:16:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:13.097375+00:00 (in 18.812714 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:16:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:54 UTC)" (scheduled at 2023-08-08 06:16:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:16:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:16:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:16:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:16:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:36.579841+00:00 (in 42.185704 seconds) namespacegcworker stdout | 2023-08-08 06:16:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:54 UTC)" (scheduled at 2023-08-08 06:16:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:16:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:16:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 54, 394518), True, datetime.datetime(2023, 8, 8, 6, 16, 54, 394518), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:16:54,401 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:16:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:16:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:17:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:16:55,139 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:16:55,143 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:16:55,145 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:16:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:16:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:02.178530+00:00 (in 6.806332 seconds) notificationworker stdout | 2023-08-08 06:16:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:05 UTC)" (scheduled at 2023-08-08 06:16:55.371682+00:00) notificationworker stdout | 2023-08-08 06:16:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:16:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 16, 55, 372686), True, datetime.datetime(2023, 8, 8, 6, 16, 55, 372686), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:16:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:16:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:16:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:16:55,688 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:16:55,989 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:16:56,185 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:16:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:16:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:19.687363+00:00 (in 22.796096 seconds) securityworker stdout | 2023-08-08 06:16:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:26 UTC)" (scheduled at 2023-08-08 06:16:56.890741+00:00) securityworker stdout | 2023-08-08 06:16:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:16:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:16:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:16:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:16:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:16:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:16:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:16:56,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:16:56,898 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:16:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:16:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 11, 56, 897776), 1, 2]) securityworker stdout | 2023-08-08 06:16:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:56,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 11, 56, 897776), 1, 2]) securityworker stdout | 2023-08-08 06:16:56,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:16:56,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:16:56,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:16:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:16:56,911 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:16:56,911 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:16:56,911 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:16:56,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:16:56,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:16:56,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:16:57,141 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:16:57,549 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:16:57,662 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:16:57,718 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:16:57,749 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:16:57,752 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:16:57,817 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:16:57,821 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} buildlogsarchiver stdout | 2023-08-08 06:16:59,174 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:16:59,325 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:16:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:16:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:09.417660+00:00 (in 9.999502 seconds) gcworker stdout | 2023-08-08 06:16:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:09 UTC)" (scheduled at 2023-08-08 06:16:59.417660+00:00) gcworker stdout | 2023-08-08 06:16:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265819418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:16:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:16:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:16:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:17:00,406 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:17:01,290 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:17:01,389 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:17:01,859 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:17:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:17:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:05.371682+00:00 (in 3.192644 seconds) notificationworker stdout | 2023-08-08 06:17:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:02 UTC)" (scheduled at 2023-08-08 06:17:02.178530+00:00) notificationworker stdout | 2023-08-08 06:17:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:17:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:17:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:17:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:12.293757+00:00 (in 9.999534 seconds) chunkcleanupworker stdout | 2023-08-08 06:17:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:12 UTC)" (scheduled at 2023-08-08 06:17:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:17:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:17:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 2, 294762), True, datetime.datetime(2023, 8, 8, 6, 17, 2, 294762), 0, 'chunk_cleanup/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:17:02,297 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} chunkcleanupworker stdout | 2023-08-08 06:17:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:17:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:17:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:12 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:17:02,659 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:17:02,998 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:17:03,434 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} builder stdout | 2023-08-08 06:17:03,438 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue gunicorn-registry stdout | 2023-08-08 06:17:03,442 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:17:03,445 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} builder stdout | 2023-08-08 06:17:03,447 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:17:03,447 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:17:03,452 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2023-08-08 06:17:03,453 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:17:03,456 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:17:03,461 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:17:03,464 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:17:03,466 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:17:03,472 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} repositorygcworker stdout | 2023-08-08 06:17:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:17:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:09.793658+00:00 (in 5.605703 seconds) repositorygcworker stdout | 2023-08-08 06:17:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:04 UTC)" (scheduled at 2023-08-08 06:17:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:17:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:17:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:17:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:17:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:15.371682+00:00 (in 9.999427 seconds) notificationworker stdout | 2023-08-08 06:17:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:15 UTC)" (scheduled at 2023-08-08 06:17:05.371682+00:00) notificationworker stdout | 2023-08-08 06:17:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:17:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 5, 372843), True, datetime.datetime(2023, 8, 8, 6, 17, 5, 372843), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:17:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:17:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:17:07,179 [200] [DEBUG] [app] Starting request: urn:request:5057deb1-5c2a-4de9-80ca-31132782b5fb (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:17:07,179 [198] [DEBUG] [app] Starting request: urn:request:2f1a23a8-4858-4181-87c5-ab3d10fe8863 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:17:07,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:07,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:07,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:17:07,184 [203] [DEBUG] [app] Starting request: urn:request:474f79ee-1454-4bfe-97e1-05fc130b3b74 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:17:07,184 [205] [DEBUG] [app] Starting request: urn:request:4f222f8c-61a1-49a5-bb51-8660ef312819 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:17:07,185 [203] [DEBUG] [app] Ending request: urn:request:474f79ee-1454-4bfe-97e1-05fc130b3b74 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:474f79ee-1454-4bfe-97e1-05fc130b3b74', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:17:07,185 [205] [DEBUG] [app] Ending request: urn:request:4f222f8c-61a1-49a5-bb51-8660ef312819 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4f222f8c-61a1-49a5-bb51-8660ef312819', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:17:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:17:07,185 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:17:07,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:07,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:07,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:07,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:07,190 [198] [DEBUG] [app] Starting request: urn:request:ed0de0ac-b096-41be-b587-a1f0bb81898e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:17:07,191 [198] [DEBUG] [app] Ending request: urn:request:ed0de0ac-b096-41be-b587-a1f0bb81898e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ed0de0ac-b096-41be-b587-a1f0bb81898e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:17:07,191 [197] [DEBUG] [app] Starting request: urn:request:86d771b7-9fc7-494c-8e09-fed60a660c1f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:17:07,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:17:07,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:07,192 [197] [DEBUG] [app] Ending request: urn:request:86d771b7-9fc7-494c-8e09-fed60a660c1f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:86d771b7-9fc7-494c-8e09-fed60a660c1f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:17:07,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:17:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:17:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:17:07,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:17:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:17:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:17:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:37.197412+00:00 (in 29.999513 seconds) buildlogsarchiver stdout | 2023-08-08 06:17:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:37 UTC)" (scheduled at 2023-08-08 06:17:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:17:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 17, 7, 198331), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:07,202 [198] [DEBUG] [app] Ending request: urn:request:2f1a23a8-4858-4181-87c5-ab3d10fe8863 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2f1a23a8-4858-4181-87c5-ab3d10fe8863', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:17:07,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:07,203 [200] [DEBUG] [app] Ending request: urn:request:5057deb1-5c2a-4de9-80ca-31132782b5fb (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5057deb1-5c2a-4de9-80ca-31132782b5fb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:17:07,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:07,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:17:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:17:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:17:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:17:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:17:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:17:07,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:17:07,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:17:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:17:07,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:17:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:17:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:17:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:19.417660+00:00 (in 9.999480 seconds) gcworker stdout | 2023-08-08 06:17:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:19 UTC)" (scheduled at 2023-08-08 06:17:09.417660+00:00) gcworker stdout | 2023-08-08 06:17:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265829418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:17:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:17:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:17:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:17:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:17:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:04.187429+00:00 (in 54.393251 seconds) repositorygcworker stdout | 2023-08-08 06:17:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:09 UTC)" (scheduled at 2023-08-08 06:17:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:17:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:17:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 9, 794560), True, datetime.datetime(2023, 8, 8, 6, 17, 9, 794560), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:17:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:17:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:17:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:17:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:17:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:22.293757+00:00 (in 9.999490 seconds) chunkcleanupworker stdout | 2023-08-08 06:17:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:22 UTC)" (scheduled at 2023-08-08 06:17:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:17:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:17:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 12, 294662), True, datetime.datetime(2023, 8, 8, 6, 17, 12, 294662), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:17:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:17:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:17:12,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:17:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:17:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:54.284093+00:00 (in 41.186196 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:17:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:13 UTC)" (scheduled at 2023-08-08 06:17:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:17:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:17:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 13, 98280), True, datetime.datetime(2023, 8, 8, 6, 17, 13, 98280), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:17:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:17:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:17:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:13 UTC)" executed successfully builder stdout | 2023-08-08 06:17:13,462 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:17:13,471 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:17:13,471 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:17:13,475 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:17:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:25.371682+00:00 (in 9.999381 seconds) notificationworker stdout | 2023-08-08 06:17:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:25 UTC)" (scheduled at 2023-08-08 06:17:15.371682+00:00) notificationworker stdout | 2023-08-08 06:17:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:17:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 15, 372859), True, datetime.datetime(2023, 8, 8, 6, 17, 15, 372859), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:17:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:17:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:17:18,428 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:17:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:17:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:29.417660+00:00 (in 9.999461 seconds) gcworker stdout | 2023-08-08 06:17:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:29 UTC)" (scheduled at 2023-08-08 06:17:19.417660+00:00) gcworker stdout | 2023-08-08 06:17:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265839418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:17:19,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:17:19,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:17:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:17:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:17:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:26.890741+00:00 (in 7.202846 seconds) securityworker stdout | 2023-08-08 06:17:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:49 UTC)" (scheduled at 2023-08-08 06:17:19.687363+00:00) securityworker stdout | 2023-08-08 06:17:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:17:19,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:17:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:17:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:17:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:17:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:17:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 12, 19, 692483), 1, 2]) securityworker stdout | 2023-08-08 06:17:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 12, 19, 692483), 1, 2]) securityworker stdout | 2023-08-08 06:17:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:17:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:17:19,722 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:17:20,737 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:17:21,195 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:17:22,179 [197] [DEBUG] [app] Starting request: urn:request:6e089ff5-a391-4784-84e3-526c1340d46e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:17:22,179 [200] [DEBUG] [app] Starting request: urn:request:d17ccc45-2ec6-4f35-9c53-3b07af19c329 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:17:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:17:22,185 [209] [DEBUG] [app] Starting request: urn:request:6c5b1c82-85e2-480f-bf86-d11a7e040d3a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:17:22,185 [207] [DEBUG] [app] Starting request: urn:request:2a8247f4-b5ee-4e3b-ade7-b52c5a0d426e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:17:22,185 [209] [DEBUG] [app] Ending request: urn:request:6c5b1c82-85e2-480f-bf86-d11a7e040d3a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6c5b1c82-85e2-480f-bf86-d11a7e040d3a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:17:22,186 [207] [DEBUG] [app] Ending request: urn:request:2a8247f4-b5ee-4e3b-ade7-b52c5a0d426e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2a8247f4-b5ee-4e3b-ade7-b52c5a0d426e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:17:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:17:22,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:17:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:22,191 [200] [DEBUG] [app] Starting request: urn:request:05a54dd0-1081-4734-8f61-6472219ab172 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:17:22,191 [200] [DEBUG] [app] Ending request: urn:request:05a54dd0-1081-4734-8f61-6472219ab172 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:05a54dd0-1081-4734-8f61-6472219ab172', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:17:22,192 [199] [DEBUG] [app] Starting request: urn:request:86a6f650-d34d-4686-a8f7-4f6ac330cba6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:17:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:17:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:22,192 [199] [DEBUG] [app] Ending request: urn:request:86a6f650-d34d-4686-a8f7-4f6ac330cba6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:86a6f650-d34d-4686-a8f7-4f6ac330cba6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:17:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:17:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:17:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:17:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:17:22,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:17:22,203 [200] [DEBUG] [app] Ending request: urn:request:d17ccc45-2ec6-4f35-9c53-3b07af19c329 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d17ccc45-2ec6-4f35-9c53-3b07af19c329', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:17:22,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:22,203 [197] [DEBUG] [app] Ending request: urn:request:6e089ff5-a391-4784-84e3-526c1340d46e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6e089ff5-a391-4784-84e3-526c1340d46e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:17:22,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:17:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:17:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:17:22,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:17:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:17:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:17:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:17:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:17:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:28.874906+00:00 (in 6.580659 seconds) chunkcleanupworker stdout | 2023-08-08 06:17:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:32 UTC)" (scheduled at 2023-08-08 06:17:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:17:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:17:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 22, 294594), True, datetime.datetime(2023, 8, 8, 6, 17, 22, 294594), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:17:22,301 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:17:22,301 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:17:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:32 UTC)" executed successfully builder stdout | 2023-08-08 06:17:23,476 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:17:23,485 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:17:23,485 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:17:23,489 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:17:23,899 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:17:25,149 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:17:25,153 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:17:25,156 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:17:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:17:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:35.371682+00:00 (in 9.999448 seconds) notificationworker stdout | 2023-08-08 06:17:25,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:35 UTC)" (scheduled at 2023-08-08 06:17:25.371682+00:00) notificationworker stdout | 2023-08-08 06:17:25,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:17:25,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 25, 373419), True, datetime.datetime(2023, 8, 8, 6, 17, 25, 373419), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:17:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:17:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:17:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:17:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:47.398551+00:00 (in 21.883643 seconds) exportactionlogsworker stdout | 2023-08-08 06:17:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:25 UTC)" (scheduled at 2023-08-08 06:17:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:17:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:17:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:17:25,697 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:17:26,026 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:17:26,212 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:17:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:17:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:49.687363+00:00 (in 22.796070 seconds) securityworker stdout | 2023-08-08 06:17:26,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:56 UTC)" (scheduled at 2023-08-08 06:17:26.890741+00:00) securityworker stdout | 2023-08-08 06:17:26,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:17:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:17:26,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:17:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:17:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:17:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:17:26,894 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:17:26,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:17:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:17:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:17:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 12, 26, 897541), 1, 2]) securityworker stdout | 2023-08-08 06:17:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 12, 26, 897541), 1, 2]) securityworker stdout | 2023-08-08 06:17:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:17:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:17:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:17:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:17:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:17:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:17:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:17:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:17:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:17:27,179 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:17:27,558 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:17:27,675 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:17:27,731 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:17:27,763 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:17:27,767 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:17:27,829 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:17:27,858 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:17:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:17:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:32.293757+00:00 (in 3.418322 seconds) chunkcleanupworker stdout | 2023-08-08 06:17:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:28 UTC)" (scheduled at 2023-08-08 06:17:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:17:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:17:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:28 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:17:29,212 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} gunicorn-web stdout | 2023-08-08 06:17:29,303 [198] [DEBUG] [app] Starting request: urn:request:1a35f7e8-ab7a-490d-9145-746320c746f6 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:29,303 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,303 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,304 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,311 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:29,311 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:29,311 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,311 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:29,311 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,311 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,311 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,311 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,314 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:29,320 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,321 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,323 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,327 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,329 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,330 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:29,332 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,332 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,332 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,332 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:29,333 [198] [DEBUG] [app] Ending request: urn:request:1a35f7e8-ab7a-490d-9145-746320c746f6 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:1a35f7e8-ab7a-490d-9145-746320c746f6', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': '', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:29,334 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:29 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.033 1121 0.033) gunicorn-web stdout | 2023-08-08 06:17:29,334 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:29 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:17:29,351 [198] [DEBUG] [app] Starting request: urn:request:6f5937e9-1c76-4e73-a8fc-efa8fe6537de (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:29,351 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,351 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,352 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,358 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:29,358 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:29,358 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,359 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:29,359 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,359 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,359 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,359 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') builder stdout | 2023-08-08 06:17:29,363 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gunicorn-web stdout | 2023-08-08 06:17:29,363 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:29,369 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,370 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,373 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,376 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,379 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,380 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:29,382 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,382 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,382 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,382 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:29,383 [198] [DEBUG] [app] Ending request: urn:request:6f5937e9-1c76-4e73-a8fc-efa8fe6537de (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:6f5937e9-1c76-4e73-a8fc-efa8fe6537de', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'D', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:29,384 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:29,384 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:29 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:29 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.035 216 0.035) gcworker stdout | 2023-08-08 06:17:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:17:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:39.417660+00:00 (in 9.999462 seconds) gcworker stdout | 2023-08-08 06:17:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:39 UTC)" (scheduled at 2023-08-08 06:17:29.417660+00:00) gcworker stdout | 2023-08-08 06:17:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265849418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:17:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:17:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:17:29,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:39 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:17:29,690 [197] [DEBUG] [app] Starting request: urn:request:68a74493-6268-41e9-afee-746d06e31e65 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:29,690 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,690 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,691 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,698 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:29,698 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:29,698 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,698 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:29,698 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,699 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,699 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,699 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,701 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:29,707 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,707 [200] [DEBUG] [app] Starting request: urn:request:5546696c-0534-4de3-86a1-17e96755753e (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:29,707 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,707 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,708 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,708 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,711 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,713 [199] [DEBUG] [app] Starting request: urn:request:9d9ca161-4bad-4545-a5b5-62d1513b5d97 (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:29,713 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,713 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,714 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,714 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:29,714 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:29,714 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:29,714 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,715 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,715 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,715 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,715 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,716 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,716 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:29,717 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:29,719 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,719 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,719 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,719 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:29,719 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,719 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,719 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,719 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,719 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,719 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:29,719 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:29,719 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:29,719 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,720 [197] [DEBUG] [app] Ending request: urn:request:68a74493-6268-41e9-afee-746d06e31e65 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:68a74493-6268-41e9-afee-746d06e31e65', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Do', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:29,720 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,720 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,720 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,720 [200] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:29,720 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,720 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:29,721 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:29 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:29 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.033 217 0.032) gunicorn-web stdout | 2023-08-08 06:17:29,721 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:29,723 [200] [DEBUG] [app] Ending request: urn:request:5546696c-0534-4de3-86a1-17e96755753e (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:5546696c-0534-4de3-86a1-17e96755753e', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:29,724 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,724 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,724 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:29,724 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:29,724 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,724 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:29,724 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:29,724 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:29 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:29 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 428 0.019) gunicorn-web stdout | 2023-08-08 06:17:29,725 [199] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:29,728 [199] [DEBUG] [app] Ending request: urn:request:9d9ca161-4bad-4545-a5b5-62d1513b5d97 (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:9d9ca161-4bad-4545-a5b5-62d1513b5d97', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:29,729 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:29,729 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:29 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:29 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 84 0.017) gunicorn-web stdout | 2023-08-08 06:17:30,036 [200] [DEBUG] [app] Starting request: urn:request:86545fbc-7035-4bae-b85d-2b79aea37bc6 (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:30,037 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,037 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,037 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,044 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:30,044 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:30,044 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,044 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,044 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,044 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,045 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,046 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:30,048 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,048 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,049 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,049 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:30,049 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,049 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,050 [200] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:30,053 [200] [DEBUG] [app] Ending request: urn:request:86545fbc-7035-4bae-b85d-2b79aea37bc6 (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:86545fbc-7035-4bae-b85d-2b79aea37bc6', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:30,054 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:30,054 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:30 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:30 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.019 84 0.019) gunicorn-web stdout | 2023-08-08 06:17:30,180 [199] [DEBUG] [app] Starting request: urn:request:11dddd68-b611-4218-909d-29ef938a5dba (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:30,180 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,180 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,181 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,188 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:30,188 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:30,188 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,188 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:30,188 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,188 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,189 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,189 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,192 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:30,198 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,202 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,205 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,208 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,209 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:30,211 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,212 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,212 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,212 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:30,212 [199] [DEBUG] [app] Ending request: urn:request:11dddd68-b611-4218-909d-29ef938a5dba (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:11dddd68-b611-4218-909d-29ef938a5dba', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Doc', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:30,213 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:30 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.034 218 0.035) gunicorn-web stdout | 2023-08-08 06:17:30,213 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:30 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityworker stdout | 2023-08-08 06:17:30,443 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gunicorn-web stdout | 2023-08-08 06:17:30,540 [200] [DEBUG] [app] Starting request: urn:request:4e518925-09bf-4eeb-9ccb-97895216e9f9 (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:30,540 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,540 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,541 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,548 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:30,548 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:30,548 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,548 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,548 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,548 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,548 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,550 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:30,552 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,553 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,553 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,553 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:30,553 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,553 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,554 [200] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:30,557 [200] [DEBUG] [app] Ending request: urn:request:4e518925-09bf-4eeb-9ccb-97895216e9f9 (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:4e518925-09bf-4eeb-9ccb-97895216e9f9', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:30,558 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:30,559 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:30 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:30 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 427 0.020) gunicorn-web stdout | 2023-08-08 06:17:30,598 [200] [DEBUG] [app] Starting request: urn:request:7164ed62-da5a-4265-bc64-8647c456a9d1 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:30,598 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,598 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,599 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,605 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:30,605 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:30,605 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,605 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:30,605 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,605 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,605 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,605 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,608 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:30,613 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,614 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,618 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,621 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,624 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,625 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:30,627 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,627 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,627 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,627 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:30,627 [200] [DEBUG] [app] Ending request: urn:request:7164ed62-da5a-4265-bc64-8647c456a9d1 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:7164ed62-da5a-4265-bc64-8647c456a9d1', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Dock', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:30,628 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:30,629 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:30 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:30 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.033 219 0.032) gunicorn-web stdout | 2023-08-08 06:17:30,808 [199] [DEBUG] [app] Starting request: urn:request:02e66774-3916-42fc-b081-f0354326b4d4 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:30,808 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,808 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,809 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,814 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:30,815 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:30,815 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,815 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:30,815 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,815 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,815 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,815 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,819 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:30,824 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,826 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,829 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,833 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,836 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,838 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:30,840 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,840 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,840 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,840 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:30,841 [199] [DEBUG] [app] Ending request: urn:request:02e66774-3916-42fc-b081-f0354326b4d4 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:02e66774-3916-42fc-b081-f0354326b4d4', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Docke', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:30,842 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:30 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.036 220 0.036) gunicorn-web stdout | 2023-08-08 06:17:30,842 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:30 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:17:30,903 [200] [DEBUG] [app] Starting request: urn:request:dd521cb8-afce-415e-b2a1-6b5a83121d53 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:30,903 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,903 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,904 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,910 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:30,910 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:30,910 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,911 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:30,911 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,911 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,911 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,911 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,914 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:30,920 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,921 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,925 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,928 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,931 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,933 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:30,934 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,935 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,935 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,935 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:30,936 [200] [DEBUG] [app] Ending request: urn:request:dd521cb8-afce-415e-b2a1-6b5a83121d53 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:dd521cb8-afce-415e-b2a1-6b5a83121d53', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Docker', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:30,937 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:30 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.036 221 0.037) gunicorn-web stdout | 2023-08-08 06:17:30,937 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:30 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:17:30,941 [200] [DEBUG] [app] Starting request: urn:request:4fcf3f0c-33b5-42b4-87ab-2a99d4a3d20e (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:30,942 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,942 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,943 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:30,949 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:30,949 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:30,949 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,949 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,949 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,949 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,949 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,951 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:30,953 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,954 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,954 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:30,954 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:30,954 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,954 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:30,955 [200] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:30,958 [200] [DEBUG] [app] Ending request: urn:request:4fcf3f0c-33b5-42b4-87ab-2a99d4a3d20e (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:4fcf3f0c-33b5-42b4-87ab-2a99d4a3d20e', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:30,959 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:30 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.019 84 0.020) gunicorn-web stdout | 2023-08-08 06:17:30,959 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:30 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:17:31,161 [199] [DEBUG] [app] Starting request: urn:request:6010d8fd-b480-408f-985f-62f56515668b (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,162 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,162 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,163 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,169 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,169 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,169 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,169 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,169 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,169 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,169 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,171 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,173 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,173 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,173 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,173 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,173 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,173 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,174 [199] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:31,178 [199] [DEBUG] [app] Ending request: urn:request:6010d8fd-b480-408f-985f-62f56515668b (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:6010d8fd-b480-408f-985f-62f56515668b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,179 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:31,179 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.019 84 0.018) gunicorn-web stdout | 2023-08-08 06:17:31,254 [198] [DEBUG] [app] Starting request: urn:request:79652c32-638e-4b13-9ed7-5ffc1aa2218f (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,255 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,255 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,255 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,261 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,261 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,261 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,262 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,262 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,262 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,262 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,263 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,265 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,266 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,266 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,266 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,266 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,266 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,267 [198] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:31,270 [198] [DEBUG] [app] Ending request: urn:request:79652c32-638e-4b13-9ed7-5ffc1aa2218f (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:79652c32-638e-4b13-9ed7-5ffc1aa2218f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,271 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:31,271 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 84 0.018) exportactionlogsworker stdout | 2023-08-08 06:17:31,298 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} gunicorn-web stdout | 2023-08-08 06:17:31,320 [200] [DEBUG] [app] Starting request: urn:request:9d8323fd-6f94-44f8-a01c-54b40804d447 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,321 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,321 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,321 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,328 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,328 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,328 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,329 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:31,329 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,329 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,329 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,329 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,331 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:31,337 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,338 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,341 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,344 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,346 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,347 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,349 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,349 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,349 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,349 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,350 [200] [DEBUG] [app] Ending request: urn:request:9d8323fd-6f94-44f8-a01c-54b40804d447 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:9d8323fd-6f94-44f8-a01c-54b40804d447', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Dockerf', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,350 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:31,351 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.032 222 0.031) servicekey stdout | 2023-08-08 06:17:31,428 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} gunicorn-web stdout | 2023-08-08 06:17:31,447 [198] [DEBUG] [app] Starting request: urn:request:491fe8d8-b5ab-410e-9ccc-e6dfe838aedb (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,448 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,448 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,448 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,454 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,454 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,454 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,454 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:31,454 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,454 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,454 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,455 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,457 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:31,462 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,463 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,466 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,469 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,471 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,472 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,474 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,474 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,474 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,474 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,475 [198] [DEBUG] [app] Ending request: urn:request:491fe8d8-b5ab-410e-9ccc-e6dfe838aedb (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:491fe8d8-b5ab-410e-9ccc-e6dfe838aedb', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Dockerfi', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,475 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 223 0.030) gunicorn-web stdout | 2023-08-08 06:17:31,476 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:17:31,614 [200] [DEBUG] [app] Starting request: urn:request:36878cab-f317-4b81-b92c-5fa4842c9315 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,614 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,614 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,615 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,621 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,621 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,621 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,622 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:31,622 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,622 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,622 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,622 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,624 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:31,630 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,631 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,633 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,636 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,639 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,640 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,642 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,642 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,642 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,642 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,642 [200] [DEBUG] [app] Ending request: urn:request:36878cab-f317-4b81-b92c-5fa4842c9315 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:36878cab-f317-4b81-b92c-5fa4842c9315', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Dockerfil', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,643 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:31,643 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 569 0.031) gunicorn-web stdout | 2023-08-08 06:17:31,678 [200] [DEBUG] [app] Starting request: urn:request:446c7a35-796c-41ff-a8aa-dda7641e4ea2 (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,679 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,679 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,679 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,685 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,685 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,685 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,686 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,686 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,686 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,686 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,687 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,690 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,690 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,690 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,690 [200] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,690 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,690 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,691 [200] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:31,695 [200] [DEBUG] [app] Ending request: urn:request:446c7a35-796c-41ff-a8aa-dda7641e4ea2 (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:446c7a35-796c-41ff-a8aa-dda7641e4ea2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,695 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:31,696 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.019 84 0.019) gunicorn-web stdout | 2023-08-08 06:17:31,752 [199] [DEBUG] [app] Starting request: urn:request:fca56ada-c0c4-4574-a8bc-6d827fb7a994 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,753 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,753 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,753 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,759 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,759 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,759 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,760 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:31,760 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,760 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,760 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,760 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,762 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:31,768 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,769 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,772 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,774 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,777 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,778 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,780 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,780 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,780 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,780 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,781 [199] [DEBUG] [app] Ending request: urn:request:fca56ada-c0c4-4574-a8bc-6d827fb7a994 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:fca56ada-c0c4-4574-a8bc-6d827fb7a994', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Dockerfile', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,781 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 225 0.031) gunicorn-web stdout | 2023-08-08 06:17:31,782 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:17:31,794 [199] [DEBUG] [app] Starting request: urn:request:efa32d1e-b4d3-42fd-86fa-7373c8aeb261 (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,794 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,794 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,795 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,800 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,800 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,801 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,801 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,801 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,801 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,801 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,802 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,805 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,805 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,805 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,805 [199] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,805 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,805 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,806 [199] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:31,809 [199] [DEBUG] [app] Ending request: urn:request:efa32d1e-b4d3-42fd-86fa-7373c8aeb261 (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:efa32d1e-b4d3-42fd-86fa-7373c8aeb261', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,809 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:31,810 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 84 0.017) globalpromstats stdout | 2023-08-08 06:17:31,895 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} gunicorn-web stdout | 2023-08-08 06:17:31,960 [198] [DEBUG] [app] Starting request: urn:request:62378c35-4b4d-4ed4-a0b9-3a0a1822be71 (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:31,960 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,960 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,961 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:31,967 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:31,967 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:31,967 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,967 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,967 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,967 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,967 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,969 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:31,971 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,972 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,972 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:31,972 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:31,972 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,972 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:31,973 [198] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:31,976 [198] [DEBUG] [app] Ending request: urn:request:62378c35-4b4d-4ed4-a0b9-3a0a1822be71 (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:62378c35-4b4d-4ed4-a0b9-3a0a1822be71', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:31,976 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 84 0.018) gunicorn-web stdout | 2023-08-08 06:17:31,977 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:31 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:17:32,108 [197] [DEBUG] [app] Starting request: urn:request:fb4fc04b-a610-42c3-8f34-dc36879606ee (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:32,109 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:32,109 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:32,109 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:32,116 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:32,116 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:32,116 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:32,117 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:32,117 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:32,117 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:32,117 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:32,118 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:32,121 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:32,121 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:32,121 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:32,121 [197] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:32,121 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:32,121 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:32,122 [197] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:32,126 [197] [DEBUG] [app] Ending request: urn:request:fb4fc04b-a610-42c3-8f34-dc36879606ee (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:fb4fc04b-a610-42c3-8f34-dc36879606ee', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:32,127 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:32,127 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:32 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:32 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 84 0.020) chunkcleanupworker stdout | 2023-08-08 06:17:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:17:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:42.293757+00:00 (in 9.999408 seconds) chunkcleanupworker stdout | 2023-08-08 06:17:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:42 UTC)" (scheduled at 2023-08-08 06:17:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:17:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:17:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 32, 294898), True, datetime.datetime(2023, 8, 8, 6, 17, 32, 294898), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:17:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:17:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:17:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:42 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:17:32,337 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:17:32,695 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:17:33,009 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:17:33,443 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:17:33,452 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:17:33,455 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:17:33,462 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:17:33,465 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:17:33,473 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:17:33,475 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:17:33,478 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:17:33,482 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} builder stdout | 2023-08-08 06:17:33,492 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:17:33,501 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:17:33,501 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:17:33,506 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:17:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:45.371682+00:00 (in 9.999469 seconds) notificationworker stdout | 2023-08-08 06:17:35,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:45 UTC)" (scheduled at 2023-08-08 06:17:35.371682+00:00) notificationworker stdout | 2023-08-08 06:17:35,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:17:35,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 35, 373173), True, datetime.datetime(2023, 8, 8, 6, 17, 35, 373173), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:17:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:17:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:35,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:17:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:17:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:54.393627+00:00 (in 17.813293 seconds) namespacegcworker stdout | 2023-08-08 06:17:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:36 UTC)" (scheduled at 2023-08-08 06:17:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:17:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:17:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:17:36,940 [198] [DEBUG] [app] Starting request: urn:request:1d12c4c7-5e94-401f-b2c7-baaf63494169 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:36,940 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:36,940 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:36,941 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:36,948 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:36,948 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:36,948 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:36,948 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:36,948 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:36,948 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:36,948 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:36,948 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:36,951 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:36,957 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:36,958 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:36,960 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:36,963 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:36,966 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:36,967 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:36,969 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:36,969 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:36,969 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:36,969 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:36,969 [198] [DEBUG] [app] Ending request: urn:request:1d12c4c7-5e94-401f-b2c7-baaf63494169 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:1d12c4c7-5e94-401f-b2c7-baaf63494169', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': 'Dockerfile', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:36,970 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:36,971 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:36 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/1.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:36 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/analyze HTTP/2.0" 200 29 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.032 565 0.032) gunicorn-web stdout | 2023-08-08 06:17:37,179 [200] [DEBUG] [app] Starting request: urn:request:2a6accf1-dc66-426b-a0bc-a750011d375e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:17:37,179 [197] [DEBUG] [app] Starting request: urn:request:a7528f1c-7ef8-4684-9008-e827bae1f9c3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:17:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:17:37,185 [207] [DEBUG] [app] Starting request: urn:request:e6df91a9-ef58-4868-b09b-9c01825c893d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:17:37,185 [210] [DEBUG] [app] Starting request: urn:request:43aad6e2-b03b-4af2-af12-f016bd074933 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:17:37,185 [207] [DEBUG] [app] Ending request: urn:request:e6df91a9-ef58-4868-b09b-9c01825c893d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e6df91a9-ef58-4868-b09b-9c01825c893d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:17:37,185 [210] [DEBUG] [app] Ending request: urn:request:43aad6e2-b03b-4af2-af12-f016bd074933 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:43aad6e2-b03b-4af2-af12-f016bd074933', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:17:37,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:17:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:17:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:37,191 [200] [DEBUG] [app] Starting request: urn:request:c1cba404-d2db-482e-a819-64bef4f73cc9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:17:37,191 [200] [DEBUG] [app] Ending request: urn:request:c1cba404-d2db-482e-a819-64bef4f73cc9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c1cba404-d2db-482e-a819-64bef4f73cc9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:17:37,192 [198] [DEBUG] [app] Starting request: urn:request:fd163098-439c-4295-b7dc-52e92cef9c49 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:17:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:17:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:37,192 [198] [DEBUG] [app] Ending request: urn:request:fd163098-439c-4295-b7dc-52e92cef9c49 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fd163098-439c-4295-b7dc-52e92cef9c49', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:17:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:17:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:17:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:17:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:17:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:17:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:17:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:07.197412+00:00 (in 29.999567 seconds) buildlogsarchiver stdout | 2023-08-08 06:17:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:07 UTC)" (scheduled at 2023-08-08 06:17:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:17:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 17, 37, 198266), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:37,203 [200] [DEBUG] [app] Ending request: urn:request:2a6accf1-dc66-426b-a0bc-a750011d375e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2a6accf1-dc66-426b-a0bc-a750011d375e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:17:37,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:17:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:17:37,203 [197] [DEBUG] [app] Ending request: urn:request:a7528f1c-7ef8-4684-9008-e827bae1f9c3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a7528f1c-7ef8-4684-9008-e827bae1f9c3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:17:37,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:17:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:17:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:17:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:17:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:17:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:17:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:17:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:17:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:17:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:17:37,304 [198] [DEBUG] [app] Starting request: urn:request:5cd15403-f8f0-406d-8351-2b5494983589 (/api/v1/organization/build_test/robots) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:37,305 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:37,305 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:37,306 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:37,312 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:37,312 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:37,312 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:37,312 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:37,312 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:37,313 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:37,313 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:37,314 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2023-08-08 06:17:37,317 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:37,317 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:17:37,317 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:17:37,317 [198] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2023-08-08 06:17:37,317 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:37,317 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:37,318 [198] [DEBUG] [peewee] ('SELECT "t1"."username", "t1"."creation_date", "t1"."last_accessed", "t2"."token", "t3"."description", "t3"."unstructured_json", "t4"."name", "t5"."name" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."user_id" = "t2"."robot_account_id") LEFT OUTER JOIN "repository" AS "t4" ON ("t6"."repository_id" = "t4"."id") LEFT OUTER JOIN "teammember" AS "t7" ON ("t7"."user_id" = "t1"."id") LEFT OUTER JOIN "team" AS "t5" ON ("t7"."team_id" = "t5"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s)) ORDER BY "t1"."last_accessed" DESC', [True, 'build_test+%']) gunicorn-web stdout | 2023-08-08 06:17:37,322 [198] [DEBUG] [app] Ending request: urn:request:5cd15403-f8f0-406d-8351-2b5494983589 (/api/v1/organization/build_test/robots) {'endpoint': 'api.orgrobotlist', 'request_id': 'urn:request:5cd15403-f8f0-406d-8351-2b5494983589', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/build_test/robots?permissions=true&token=false', 'path': '/api/v1/organization/build_test/robots', 'parameters': {'permissions': 'true', 'token': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:37,322 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:37,323 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:37 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:37 +0000] "GET /api/v1/organization/build_test/robots?permissions=true&token=false HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 677 0.020) gcworker stdout | 2023-08-08 06:17:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:17:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:49.417660+00:00 (in 9.999459 seconds) gcworker stdout | 2023-08-08 06:17:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:49 UTC)" (scheduled at 2023-08-08 06:17:39.417660+00:00) gcworker stdout | 2023-08-08 06:17:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265859418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:17:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:17:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:17:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:49 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:17:41,454 [200] [DEBUG] [app] Starting request: urn:request:53714c1b-ab31-4bb1-a96f-ab28c4334f59 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/activate) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:41,454 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:41,454 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:41,455 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,462 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:41,462 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:41,462 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:41,462 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:41,462 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:41,462 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:41,462 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:41,462 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:41,465 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:41,471 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:41,472 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,475 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,478 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,481 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,482 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:41,483 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['write', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,484 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,486 [200] [DEBUG] [peewee] ('INSERT INTO "accesstoken" ("friendly_name", "token_name", "token_code", "repository_id", "created", "role_id", "temporary") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "accesstoken"."id"', ['Build Trigger: custom-git', '005IJD4XUFDFD1CTSRTDNH4N4NYM53KO', 'v0$$yxlDO0JASuH0lnhkJVTfsES+FNnCfZWAvkUpb53Y7GaMi0L1Rxr/794LWQENarP9Ox9ZVN1oOE19N/YSAA==', 3, datetime.datetime(2023, 8, 8, 6, 17, 41, 486020), 2, False]) gunicorn-web stdout | 2023-08-08 06:17:41,561 [200] [DEBUG] [peewee] ('UPDATE "repositorybuildtrigger" SET "uuid" = %s, "service_id" = %s, "repository_id" = %s, "connected_user_id" = %s, "secure_auth_token" = %s, "secure_private_key" = %s, "fully_migrated" = %s, "config" = %s, "write_token_id" = %s, "pull_robot_id" = %s, "enabled" = %s, "disabled_reason_id" = %s, "disabled_datetime" = %s, "successive_failure_count" = %s, "successive_internal_error_count" = %s WHERE ("repositorybuildtrigger"."id" = %s)', ['1f727fd3-07cc-4975-bada-b401b1746237', 4, 3, 1, None, 'v0$$3HGk9HQA5PXxxb/1MWbSZhq+o16lx2WN9KkZA3tF6WUVDLgegJ/ga89aQUwvRi8ddv7VDIPykkWZKnM4v/ej3vsW7hNYkX/bMow8CxtdT/0eMDiN4NtS2pro2hLitTpC8csaOcbas7qGkjpESVjLLHHYSZ9P9KUDRcrnQSGq0T1Wg+kIirtd6AUvAN0A01pwKlYkqZUtgn2/0fqGhUhZ9OCxjbrPsUtyNTl2ojdr4EiAo/O6lQBLvN1ILRv1kSSxxAecIOrF9RHvb0m88rGIDjWEwagOqiMb6OuM72qSSoDWPtBTA5YZJIWpeHgo2DYc25dVnyshxJsgyrfe+qgthsN6szAVOV7ZvjfHsb1OTX+6SCvmMcsUHXCpgP1iRDGMSHM1Y4rIcuKIwNU/rYckO4jA/I8zmfZWbmOWxHqoqjJSN/vE18wYRVDqDEpgBD/Xrjl/s6O0PoyFZaWzWxzs6YJz7fQt4D3SmkRuUSVcE3pY8RGg25rH1TAPOh6VBzTc8znXJriOtkvFmwRtkEXeKUI/e8gtvTrGr8qnuHX7xtIm2qmU8PiJGTkVsiKwCLLv35lwStaEQEXRv05o5B3CvGAZWHlssWfSc3Md+ihttEUklMY6ztWk6Xptx1m3jKPS8UJ56NWLtVWNfzulzORfKLP2s1hGjsr0B1cdgIemutN357LmYAdNmK9EaHGgqrJeXdY3f1o4lGRs1xrKCEaNLxmGiiSQBIZAaHIrQ0hEO+enrBwBgnUzYeC0ai4jRi6wrUv4wd2wjc/XmR4L1xuNSh2BVPhQssnLLwGe+5EYkoIhyBVqZt4ok49cciVvOXXvj/1LEVg/Evk0Y+Q/vVwKmZOPuwhqk9oCwuM7ZjMsDGlVf6DbZWyKm6EWnzoZNVtcVs3XvOk04EXhZch4TuyzsBdcMegOboygbbL+hH5xubHhteig6oQlI2p0jRVe0Ac74bFO8J8rXe7k3ONwEjFqp40PVK3kBsSS4qsc/p3evR7wShpGQKo8IJjJarFVpwKczRFV+0++Ra/wrmeNsa/rl0fjynm8QgkQwz4DxAT2HWoPfxGC7UYJwrrj/lYv7NQhLAk9tl9/g4wExKwhoxQEp73Hg7J1HoYgtwSdFuqhqjXi6n84PDbFXnot9GcmPYZBEu8diH04jli3xJiXsrfmAtYmEpaqkDGojzRE4AJhrA2OzYrq++Ntv+wu2i6VXTC/19zg52zuPUGoiFquL3fTiHZySZ9HPinvcouKyPc7PkwDEwTNX1jHIy/iX5BfeX8mRL+A5LKnxXGZ5FmWYAMufIvdWCiQGRU5TeRn4t/P/0uSP2+0yIX4mp9mpsaD6SP/mKjllinewmSyB4YZSGz6/idtzO7AqrtybAr3cI2sywiGBNqt83mqHjKxsYtJL9t7jjzfFopFrB9+w59B8U70YH9LBd2FTJl0SA6OZZm/BgpZ5jHjkIlxur8aGWjGu/YlbJh8DwFvzzEqCCJCseXv9iXCD8+edOaukX7zCR/R+9QxEyf1YPE5mNDQEDAXKlAfn47Wg3SmsSUMaKW5/aeXlnvRjVIc7V9b1agQebq0ESeRaaqc5uaCxRv4CPx0VoH9UcjyyQdLnz9QnCB6OJZ21Xh+mjnZxhWOW3OA6wj1cRendyeCeqFGxUooE9TLDxuFtEy2/5qj70u0bU5ntibqe7fvnpBM91CExv0WdfkocJ1SMjkp1nfx7xKXwUW7T25BTybPjppu4UInijMKucMDxz7rpY80XBm7Om+AiiyTFAtb9Dfs48Yrf/bdSB5kwMUKwXPSXaCJQ/G7czuCguIdgO/un74hQEW6uii9xvBpQR4pCjuBteYppDt3oim27ylqG8LqS1aHDzwHr8AP2ryhm5zFc1WvzLWpIcnxN/oBxsOGtbYHi5tIcHVVPzpleY+SElW7ALuhk/whMnvrWHxHNgIfa2McW0iOGSGFeY7TQdwAmO3HZOMm2egXC8OjUjRjsDzygpNCMF0PuJ4oCgaiN25e20HPnjlB/3K27YZNPCUY7+g25T3nfmBakWP01vR7vJcF2LCKuKk1zU7oaNbj/IDOWzn91ya/YqVu50r1WwVbCclo6GPO/vhkLAFf+LmjYsFKQVLZfd6xD7atd5CTTXuc1UtI+vHDCppRKMtHQkNrD4IPgKtbJIqA2nTGQr/8DfvcOIBRXAHdUXP7m000auavZ1iGaK6VEzalwA5IA62tQB31HR+TcKHd/hsI195a/Gx9HmPcup0lDEFxfOnYvgCyU7qP+5hN', False, '{"build_source": "git@github.com:hu-weihua/quay_build_test.git", "dockerfile_path": "/Dockerfile", "context": "/", "default_tag_from_ref": true, "latest_for_default_branch": true, "tag_templates": [], "credentials": [{"name": "SSH Public Key", "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC07IvRhMkmgmuDcWBe+RVitNU3gJ7uzgFqC7+91IK1XCxcdA8QEQXJVcqWwHpUQtO9XfxmP7tPINTMJWw0wx1wmQL0jgMjjRvVAoeHvZAUZXHEIa0G/Jy+lt+mXsELJtqY2vfDZ4Ws87Gix7nyDIvTBC/nHru21M3ZxXiIJgj09IKqbgh8QKL6SJNF0B8XKL6rHLbyXat2Fw/dINkBCKrg2gVd5mnMtq+cU0TAVRzMo3IskuAWM7EB9exyrUudISrX1NVsvjfAWlNT9cSKPprxVEgOtpXggsUe/bsbPW64criLK36ZY2vguGX4uK0FPBBtu1QQhgZChwqnTOPkHBRF"}, {"name": "Webhook Endpoint URL", "value": "https://$token:005IJD4XUFDFD1CTSRTDNH4N4NYM53KOUBH3W0X0PL1X343UHR2IWGWL65XC99B8@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/1f727fd3-07cc-4975-bada-b401b1746237"}]}', 5, None, True, None, datetime.datetime(2023, 8, 8, 6, 16, 51, 664145), 0, 0, 4]) gunicorn-web stdout | 2023-08-08 06:17:41,567 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,569 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:41,570 [200] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [53, 2, 1, 3, datetime.datetime(2023, 8, 8, 6, 17, 41, 569220), '10.130.2.1', '{"repo": "custom", "namespace": "build_test", "trigger_id": "1f727fd3-07cc-4975-bada-b401b1746237", "service": "custom-git", "pull_robot": null, "config": {"build_source": "git@github.com:hu-weihua/quay_build_test.git", "dockerfile_path": "/Dockerfile", "context": "/", "default_tag_from_ref": true, "latest_for_default_branch": true, "tag_templates": [], "credentials": [{"name": "SSH Public Key", "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC07IvRhMkmgmuDcWBe+RVitNU3gJ7uzgFqC7+91IK1XCxcdA8QEQXJVcqWwHpUQtO9XfxmP7tPINTMJWw0wx1wmQL0jgMjjRvVAoeHvZAUZXHEIa0G/Jy+lt+mXsELJtqY2vfDZ4Ws87Gix7nyDIvTBC/nHru21M3ZxXiIJgj09IKqbgh8QKL6SJNF0B8XKL6rHLbyXat2Fw/dINkBCKrg2gVd5mnMtq+cU0TAVRzMo3IskuAWM7EB9exyrUudISrX1NVsvjfAWlNT9cSKPprxVEgOtpXggsUe/bsbPW64criLK36ZY2vguGX4uK0FPBBtu1QQhgZChwqnTOPkHBRF"}, {"name": "Webhook Endpoint URL", "value": "https://$token:005IJD4XUFDFD1CTSRTDNH4N4NYM53KOUBH3W0X0PL1X343UHR2IWGWL65XC99B8@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/1f727fd3-07cc-4975-bada-b401b1746237"}]}}']) gunicorn-web stdout | 2023-08-08 06:17:41,574 [200] [DEBUG] [app] Ending request: urn:request:53714c1b-ab31-4bb1-a96f-ab28c4334f59 (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/activate) {'endpoint': 'api.buildtriggeractivate', 'request_id': 'urn:request:53714c1b-ab31-4bb1-a96f-ab28c4334f59', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/activate', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/activate', 'parameters': {}, 'json_body': {'config': {'build_source': 'git@github.com:hu-weihua/quay_build_test.git', 'dockerfile_path': '/Dockerfile', 'context': '/', 'default_tag_from_ref': True, 'latest_for_default_branch': True, 'tag_templates': [], 'credentials': [{'name': 'SSH Public Key', 'value': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC07IvRhMkmgmuDcWBe+RVitNU3gJ7uzgFqC7+91IK1XCxcdA8QEQXJVcqWwHpUQtO9XfxmP7tPINTMJWw0wx1wmQL0jgMjjRvVAoeHvZAUZXHEIa0G/Jy+lt+mXsELJtqY2vfDZ4Ws87Gix7nyDIvTBC/nHru21M3ZxXiIJgj09IKqbgh8QKL6SJNF0B8XKL6rHLbyXat2Fw/dINkBCKrg2gVd5mnMtq+cU0TAVRzMo3IskuAWM7EB9exyrUudISrX1NVsvjfAWlNT9cSKPprxVEgOtpXggsUe/bsbPW64criLK36ZY2vguGX4uK0FPBBtu1QQhgZChwqnTOPkHBRF'}, {'name': 'Webhook Endpoint URL', 'value': 'https://$token:005IJD4XUFDFD1CTSRTDNH4N4NYM53KOUBH3W0X0PL1X343UHR2IWGWL65XC99B8@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/1f727fd3-07cc-4975-bada-b401b1746237'}]}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:41,575 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:41,575 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:41 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/activate HTTP/1.0" 200 1155 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:41 +0000] "POST /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237/activate HTTP/2.0" 200 1155 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.123 968 0.122) chunkcleanupworker stdout | 2023-08-08 06:17:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:17:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:52.293757+00:00 (in 9.999405 seconds) chunkcleanupworker stdout | 2023-08-08 06:17:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:52 UTC)" (scheduled at 2023-08-08 06:17:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:17:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:17:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 42, 294865), True, datetime.datetime(2023, 8, 8, 6, 17, 42, 294865), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:17:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:17:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:17:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:52 UTC)" executed successfully builder stdout | 2023-08-08 06:17:43,516 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:17:43,525 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:17:43,525 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:17:43,530 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:17:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:55.371682+00:00 (in 9.999495 seconds) notificationworker stdout | 2023-08-08 06:17:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:55 UTC)" (scheduled at 2023-08-08 06:17:45.371682+00:00) notificationworker stdout | 2023-08-08 06:17:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:17:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 45, 372695), True, datetime.datetime(2023, 8, 8, 6, 17, 45, 372695), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:17:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:17:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:17:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:17:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:25.514341+00:00 (in 38.115198 seconds) exportactionlogsworker stdout | 2023-08-08 06:17:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:47 UTC)" (scheduled at 2023-08-08 06:17:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:17:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:17:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 47, 399750), True, datetime.datetime(2023, 8, 8, 6, 17, 47, 399750), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:17:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:17:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:17:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:17:48,443 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:17:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:17:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:59.417660+00:00 (in 9.999436 seconds) gcworker stdout | 2023-08-08 06:17:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:59 UTC)" (scheduled at 2023-08-08 06:17:49.417660+00:00) gcworker stdout | 2023-08-08 06:17:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265869418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:17:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:17:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:17:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:17:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:17:49,688 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:17:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:17:56.890741+00:00 (in 7.202233 seconds) securityworker stdout | 2023-08-08 06:17:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:19 UTC)" (scheduled at 2023-08-08 06:17:49.687363+00:00) securityworker stdout | 2023-08-08 06:17:49,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:17:49,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:17:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:17:49,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:17:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:17:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:17:49,737 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:17:50,775 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:17:51,232 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:17:52,179 [197] [DEBUG] [app] Starting request: urn:request:ab731000-1b6a-4741-86f7-fd2f6edca382 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:17:52,179 [199] [DEBUG] [app] Starting request: urn:request:c4cc642d-212e-426b-a3ca-ccedd2cb24e7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:17:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:52,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:17:52,185 [210] [DEBUG] [app] Starting request: urn:request:23e39cdb-a0f6-4638-bf5a-fa652ac7dee5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:17:52,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:17:52,186 [210] [DEBUG] [app] Ending request: urn:request:23e39cdb-a0f6-4638-bf5a-fa652ac7dee5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:23e39cdb-a0f6-4638-bf5a-fa652ac7dee5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:17:52,186 [207] [DEBUG] [app] Starting request: urn:request:e3b858ec-725d-4cc7-b284-f424491f2205 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:17:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:17:52,186 [207] [DEBUG] [app] Ending request: urn:request:e3b858ec-725d-4cc7-b284-f424491f2205 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e3b858ec-725d-4cc7-b284-f424491f2205', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:17:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:17:52,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:17:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:52,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:17:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:52,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:17:52,191 [200] [DEBUG] [app] Starting request: urn:request:8354d082-b87b-43ae-a400-8a8692d5b41e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:17:52,192 [200] [DEBUG] [app] Ending request: urn:request:8354d082-b87b-43ae-a400-8a8692d5b41e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8354d082-b87b-43ae-a400-8a8692d5b41e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:17:52,192 [199] [DEBUG] [app] Starting request: urn:request:5a12104f-0abf-4af5-aea7-8009c8614f0e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:17:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:17:52,192 [199] [DEBUG] [app] Ending request: urn:request:5a12104f-0abf-4af5-aea7-8009c8614f0e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5a12104f-0abf-4af5-aea7-8009c8614f0e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:17:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:17:52,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:17:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:17:52,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:52,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:17:52,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:17:52,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:17:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:17:52,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:17:52,205 [197] [DEBUG] [app] Ending request: urn:request:ab731000-1b6a-4741-86f7-fd2f6edca382 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ab731000-1b6a-4741-86f7-fd2f6edca382', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:17:52,205 [199] [DEBUG] [app] Ending request: urn:request:c4cc642d-212e-426b-a3ca-ccedd2cb24e7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c4cc642d-212e-426b-a3ca-ccedd2cb24e7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:17:52,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:52,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:52,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:17:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:17:52,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:17:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:17:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:17:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:17:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:17:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:02.293757+00:00 (in 9.999431 seconds) chunkcleanupworker stdout | 2023-08-08 06:17:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:02 UTC)" (scheduled at 2023-08-08 06:17:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:17:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:17:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 52, 294741), True, datetime.datetime(2023, 8, 8, 6, 17, 52, 294741), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:17:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:17:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:17:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:02 UTC)" executed successfully builder stdout | 2023-08-08 06:17:53,530 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:17:53,540 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:17:53,540 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:17:53,544 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:17:53,907 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:17:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:17:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:13.097375+00:00 (in 18.812685 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:17:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:54 UTC)" (scheduled at 2023-08-08 06:17:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:17:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:17:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:17:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:17:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:36.579841+00:00 (in 42.185602 seconds) namespacegcworker stdout | 2023-08-08 06:17:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:54 UTC)" (scheduled at 2023-08-08 06:17:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:17:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:17:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 54, 394801), True, datetime.datetime(2023, 8, 8, 6, 17, 54, 394801), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:17:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:17:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:17:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:18:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:17:55,158 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:17:55,163 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:17:55,166 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:17:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:17:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:02.178530+00:00 (in 6.806324 seconds) notificationworker stdout | 2023-08-08 06:17:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:05 UTC)" (scheduled at 2023-08-08 06:17:55.371682+00:00) notificationworker stdout | 2023-08-08 06:17:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:17:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 17, 55, 372728), True, datetime.datetime(2023, 8, 8, 6, 17, 55, 372728), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:17:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:17:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:17:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:05 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:17:55,618 [200] [DEBUG] [app] Starting request: urn:request:19dc4bf6-6e10-4fe8-9878-093d53e74212 (/api/v1/repository/build_test/custom) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:55,618 [199] [DEBUG] [app] Starting request: urn:request:89e025d1-474a-4eab-baa6-05d1af9706b2 (/api/v1/repository/build_test/custom/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:55,618 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,618 [199] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,618 [199] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,618 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,619 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:55,619 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:55,625 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:55,625 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:55,625 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,626 [199] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:55,626 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,626 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:55,626 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:55,626 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:55,626 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:55,626 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:55,626 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,626 [200] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:55,627 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,627 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:55,627 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:55,627 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:55,628 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:55,629 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:55,634 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:55,635 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:55,635 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:55,635 [200] [DEBUG] [endpoints.api.repository] Get repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:55,636 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:55,638 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [3, 1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:55,638 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:55,640 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [3, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:17:55,640 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [3, None, 1691475475640, False, 101, 0]) gunicorn-web stdout | 2023-08-08 06:17:55,641 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,641 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:55,642 [200] [DEBUG] [app] Ending request: urn:request:19dc4bf6-6e10-4fe8-9878-093d53e74212 (/api/v1/repository/build_test/custom) {'endpoint': 'api.repository', 'request_id': 'urn:request:19dc4bf6-6e10-4fe8-9878-093d53e74212', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/custom', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:55,642 [199] [DEBUG] [app] Ending request: urn:request:89e025d1-474a-4eab-baa6-05d1af9706b2 (/api/v1/repository/build_test/custom/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:89e025d1-474a-4eab-baa6-05d1af9706b2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/custom/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:55,643 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:55,643 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:55 +0000] "GET /api/v1/repository/build_test/custom?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:17:55,643 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:55 +0000] "GET /api/v1/repository/build_test/custom?includeStats=true&includeTags=false HTTP/2.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 87 0.027) gunicorn-web stdout | 2023-08-08 06:17:55,643 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:55 +0000] "GET /api/v1/repository/build_test/custom/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:55 +0000] "GET /api/v1/repository/build_test/custom/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 885 0.028) blobuploadcleanupworker stdout | 2023-08-08 06:17:55,713 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:17:56,062 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:17:56,220 [197] [DEBUG] [app] Starting request: urn:request:9ed6837b-a71a-435f-89bb-09db2616db7c (/api/v1/repository/build_test/custom/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:56,220 [198] [DEBUG] [app] Starting request: urn:request:609f24f8-6261-4430-88f6-5c04a0762d80 (/api/v1/repository/build_test/custom/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:56,220 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,220 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,220 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,221 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,221 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,222 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,227 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:56,227 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:56,227 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,227 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:56,228 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:56,228 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,228 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,228 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:56,228 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,228 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,228 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,228 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:56,228 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,228 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,228 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,228 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,230 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:56,230 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:56,236 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,237 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,237 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,238 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,240 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'custom']) gunicorn-web stdout | 2023-08-08 06:17:56,241 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['build_test', 'custom']) gunicorn-web stdout | 2023-08-08 06:17:56,242 [197] [DEBUG] [app] Ending request: urn:request:9ed6837b-a71a-435f-89bb-09db2616db7c (/api/v1/repository/build_test/custom/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:9ed6837b-a71a-435f-89bb-09db2616db7c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/', 'path': '/api/v1/repository/build_test/custom/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:56,243 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:56,244 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:56 +0000] "GET /api/v1/repository/build_test/custom/trigger/ HTTP/1.0" 200 1171 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:56 +0000] "GET /api/v1/repository/build_test/custom/trigger/ HTTP/2.0" 200 1171 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.025 412 0.025) gunicorn-web stdout | 2023-08-08 06:17:56,244 [198] [DEBUG] [app] Ending request: urn:request:609f24f8-6261-4430-88f6-5c04a0762d80 (/api/v1/repository/build_test/custom/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:609f24f8-6261-4430-88f6-5c04a0762d80', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/', 'path': '/api/v1/repository/build_test/custom/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:56,244 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:56,245 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:56 +0000] "GET /api/v1/repository/build_test/custom/trigger/ HTTP/1.0" 200 1171 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:56 +0000] "GET /api/v1/repository/build_test/custom/trigger/ HTTP/2.0" 200 1171 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 68 0.026) chunkcleanupworker stdout | 2023-08-08 06:17:56,249 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:17:56,415 [197] [DEBUG] [app] Starting request: urn:request:2fea729f-3cc4-4032-a1e0-87a2316bac37 (/repository/build_test/custom/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:56,417 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,424 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,426 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,430 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['custom', 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,436 [197] [DEBUG] [app] Ending request: urn:request:2fea729f-3cc4-4032-a1e0-87a2316bac37 (/repository/build_test/custom/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:2fea729f-3cc4-4032-a1e0-87a2316bac37', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom/status?token=701a93e9-baf8-4db2-9544-e3d916850a71', 'path': '/repository/build_test/custom/status', 'parameters': {'token': '701a93e9-baf8-4db2-9544-e3d916850a71'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:56,437 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:56,437 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:56 +0000] "GET /repository/build_test/custom/status?token=701a93e9-baf8-4db2-9544-e3d916850a71 HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:56 +0000] "GET /repository/build_test/custom/status?token=701a93e9-baf8-4db2-9544-e3d916850a71 HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.023 124 0.024) gunicorn-web stdout | 2023-08-08 06:17:56,616 [198] [DEBUG] [app] Starting request: urn:request:7ca93652-1362-4cdb-a4b2-a155def358b5 (/api/v1/repository/build_test/custom/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:56,616 [198] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,616 [198] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,617 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,624 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:56,624 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:56,624 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,624 [198] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:56,624 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:56,624 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,624 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,624 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,627 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:56,633 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:56,634 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:56,638 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['custom', 'build_test', 3]) gunicorn-web stdout | 2023-08-08 06:17:56,641 [198] [DEBUG] [app] Ending request: urn:request:7ca93652-1362-4cdb-a4b2-a155def358b5 (/api/v1/repository/build_test/custom/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:7ca93652-1362-4cdb-a4b2-a155def358b5', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/build/?limit=3', 'path': '/api/v1/repository/build_test/custom/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:56,642 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:56,642 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:56 +0000] "GET /api/v1/repository/build_test/custom/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:56 +0000] "GET /api/v1/repository/build_test/custom/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 419 0.028) securityworker stdout | 2023-08-08 06:17:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:17:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:19.687363+00:00 (in 22.796029 seconds) securityworker stdout | 2023-08-08 06:17:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:26 UTC)" (scheduled at 2023-08-08 06:17:56.890741+00:00) securityworker stdout | 2023-08-08 06:17:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:17:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:17:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:17:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:17:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:17:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:17:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:17:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:17:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:17:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:17:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 12, 56, 896677), 1, 2]) securityworker stdout | 2023-08-08 06:17:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 12, 56, 896677), 1, 2]) securityworker stdout | 2023-08-08 06:17:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:17:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:17:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:17:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:17:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:17:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:17:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:17:56,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:17:56,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:17:56,908 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:17:57,029 [197] [DEBUG] [app] Starting request: urn:request:4d2e254a-88c9-4544-b24d-b95703bb5b7b (/api/v1/repository/build_test/custom/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:17:57,029 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:57,029 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:57,030 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:57,036 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:17:57,037 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:17:57,037 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:57,037 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:17:57,037 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:17:57,037 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:57,037 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:57,037 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:17:57,039 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:17:57,046 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:17:57,047 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:17:57,051 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['custom', 'build_test', 10]) gunicorn-web stdout | 2023-08-08 06:17:57,054 [197] [DEBUG] [app] Ending request: urn:request:4d2e254a-88c9-4544-b24d-b95703bb5b7b (/api/v1/repository/build_test/custom/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:4d2e254a-88c9-4544-b24d-b95703bb5b7b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/build/?limit=10', 'path': '/api/v1/repository/build_test/custom/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:17:57,055 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:17:57,055 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:17:57 +0000] "GET /api/v1/repository/build_test/custom/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:17:57 +0000] "GET /api/v1/repository/build_test/custom/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 73 0.027) teamsyncworker stdout | 2023-08-08 06:17:57,205 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:17:57,567 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:17:57,688 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:17:57,744 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:17:57,776 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:17:57,779 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:17:57,867 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:17:57,883 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} buildlogsarchiver stdout | 2023-08-08 06:17:59,250 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:17:59,401 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gcworker stdout | 2023-08-08 06:17:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:17:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:09.417660+00:00 (in 9.999478 seconds) gcworker stdout | 2023-08-08 06:17:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:09 UTC)" (scheduled at 2023-08-08 06:17:59.417660+00:00) gcworker stdout | 2023-08-08 06:17:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265879418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:17:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:17:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:17:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:09 UTC)" executed successfully securityworker stdout | 2023-08-08 06:18:00,471 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:18:01,328 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:18:01,438 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:18:01,911 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:18:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:18:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:05.371682+00:00 (in 3.192635 seconds) notificationworker stdout | 2023-08-08 06:18:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:02 UTC)" (scheduled at 2023-08-08 06:18:02.178530+00:00) notificationworker stdout | 2023-08-08 06:18:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:18:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:18:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:18:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:12.293757+00:00 (in 9.999407 seconds) chunkcleanupworker stdout | 2023-08-08 06:18:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:12 UTC)" (scheduled at 2023-08-08 06:18:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:18:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:18:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 2, 294838), True, datetime.datetime(2023, 8, 8, 6, 18, 2, 294838), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:18:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:18:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:18:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:12 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:18:02,349 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:18:02,703 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:18:03,045 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:18:03,451 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:18:03,461 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:18:03,463 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:18:03,470 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:18:03,473 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:18:03,483 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:18:03,486 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:18:03,489 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:18:03,492 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} builder stdout | 2023-08-08 06:18:03,554 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:18:03,564 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:18:03,564 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:18:03,568 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:18:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:18:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:09.793658+00:00 (in 5.605724 seconds) repositorygcworker stdout | 2023-08-08 06:18:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:04 UTC)" (scheduled at 2023-08-08 06:18:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:18:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:18:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:18:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:18:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:15.371682+00:00 (in 9.999507 seconds) notificationworker stdout | 2023-08-08 06:18:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:15 UTC)" (scheduled at 2023-08-08 06:18:05.371682+00:00) notificationworker stdout | 2023-08-08 06:18:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:18:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 5, 372557), True, datetime.datetime(2023, 8, 8, 6, 18, 5, 372557), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:18:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:18:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:18:07,179 [200] [DEBUG] [app] Starting request: urn:request:1d1d3b10-9733-441b-8cfa-3a99fc6e36c0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:18:07,180 [198] [DEBUG] [app] Starting request: urn:request:c8e4d1bf-f9e1-469b-90a2-a2d9b89c4c41 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:18:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:07,182 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:07,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:18:07,185 [206] [DEBUG] [app] Starting request: urn:request:740a8776-ece1-487a-9551-2674eec93606 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:18:07,185 [206] [DEBUG] [app] Ending request: urn:request:740a8776-ece1-487a-9551-2674eec93606 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:740a8776-ece1-487a-9551-2674eec93606', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:18:07,185 [203] [DEBUG] [app] Starting request: urn:request:afae0653-67b7-492c-b4d6-94a553261d0c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:18:07,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:18:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:18:07,186 [203] [DEBUG] [app] Ending request: urn:request:afae0653-67b7-492c-b4d6-94a553261d0c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:afae0653-67b7-492c-b4d6-94a553261d0c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:18:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.001) gunicorn-web stdout | 2023-08-08 06:18:07,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:07,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:07,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:07,191 [197] [DEBUG] [app] Starting request: urn:request:835de29c-f065-42f4-8b6c-47214f3ff1f0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:18:07,192 [197] [DEBUG] [app] Ending request: urn:request:835de29c-f065-42f4-8b6c-47214f3ff1f0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:835de29c-f065-42f4-8b6c-47214f3ff1f0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:18:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:18:07,192 [200] [DEBUG] [app] Starting request: urn:request:f1fa9421-f6fc-4435-999f-1a68f9338d0f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:18:07,193 [200] [DEBUG] [app] Ending request: urn:request:f1fa9421-f6fc-4435-999f-1a68f9338d0f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f1fa9421-f6fc-4435-999f-1a68f9338d0f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:18:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:18:07,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:07,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:07,194 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:18:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:18:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:18:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:37.197412+00:00 (in 29.999523 seconds) buildlogsarchiver stdout | 2023-08-08 06:18:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:37 UTC)" (scheduled at 2023-08-08 06:18:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:18:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 18, 7, 198296), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:07,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:18:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:18:07,204 [198] [DEBUG] [app] Ending request: urn:request:c8e4d1bf-f9e1-469b-90a2-a2d9b89c4c41 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c8e4d1bf-f9e1-469b-90a2-a2d9b89c4c41', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:18:07,204 [198] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:18:07,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:07,204 [200] [DEBUG] [app] Ending request: urn:request:1d1d3b10-9733-441b-8cfa-3a99fc6e36c0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1d1d3b10-9733-441b-8cfa-3a99fc6e36c0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:18:07,205 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:18:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:18:07,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:18:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:18:07,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:18:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:18:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:18:07,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:18:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:18:07,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:18:07,209 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:18:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:18:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:19.417660+00:00 (in 9.999484 seconds) gcworker stdout | 2023-08-08 06:18:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:19 UTC)" (scheduled at 2023-08-08 06:18:09.417660+00:00) gcworker stdout | 2023-08-08 06:18:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265889418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:18:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:18:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:18:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:18:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:18:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:04.187429+00:00 (in 54.393245 seconds) repositorygcworker stdout | 2023-08-08 06:18:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:09 UTC)" (scheduled at 2023-08-08 06:18:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:18:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:18:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 9, 794557), True, datetime.datetime(2023, 8, 8, 6, 18, 9, 794557), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:18:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:18:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:18:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:09 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:18:12,034 [197] [DEBUG] [app] Starting request: urn:request:57a7fcb9-9f7b-4130-981d-6affeae4f2dd (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:12,034 [197] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:18:12,034 [197] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:18:12,035 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:12,041 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:12,042 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:12,042 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:18:12,042 [197] [DEBUG] [endpoints.api] Checking permission for repo: build_test/custom gunicorn-web stdout | 2023-08-08 06:18:12,042 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:18:12,042 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:18:12,042 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:18:12,042 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2023-08-08 06:18:12,045 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['custom', 'build_test', 1, 'custom', 'build_test', 1]) gunicorn-web stdout | 2023-08-08 06:18:12,051 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='build_test', name='custom', role='admin') gunicorn-web stdout | 2023-08-08 06:18:12,052 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:12,054 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:12,057 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['1f727fd3-07cc-4975-bada-b401b1746237', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:12,060 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'custom', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:12,062 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:12,063 [197] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [23, 2, 1, 3, datetime.datetime(2023, 8, 8, 6, 18, 12, 61576), '10.130.2.1', '{"repo": "custom", "trigger_id": "1f727fd3-07cc-4975-bada-b401b1746237", "service": "custom-git"}']) gunicorn-web stdout | 2023-08-08 06:18:12,068 [197] [DEBUG] [peewee] ('UPDATE "repositorybuild" SET "trigger_id" = %s WHERE ("repositorybuild"."trigger_id" = %s)', [None, 4]) gunicorn-web stdout | 2023-08-08 06:18:12,069 [197] [DEBUG] [peewee] ('DELETE FROM "repositorybuildtrigger" WHERE ("repositorybuildtrigger"."id" = %s)', [4]) gunicorn-web stdout | 2023-08-08 06:18:12,072 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."friendly_name", "t1"."token_name", "t1"."token_code", "t1"."repository_id", "t1"."created", "t1"."role_id", "t1"."temporary", "t1"."kind_id" FROM "accesstoken" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [5, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:12,073 [197] [DEBUG] [peewee] ('DELETE FROM "accesstoken" WHERE ("accesstoken"."id" = %s)', [5]) gunicorn-web stdout | 2023-08-08 06:18:12,077 [197] [DEBUG] [app] Ending request: urn:request:57a7fcb9-9f7b-4130-981d-6affeae4f2dd (/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:57a7fcb9-9f7b-4130-981d-6affeae4f2dd', 'remote_addr': '10.130.2.1', 'http_method': 'DELETE', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237', 'path': '/api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237', 'parameters': {}, 'json_body': {}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:12,078 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:12,078 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:12 +0000] "DELETE /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237 HTTP/1.0" 204 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:12 +0000] "DELETE /api/v1/repository/build_test/custom/trigger/1f727fd3-07cc-4975-bada-b401b1746237 HTTP/2.0" 204 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.046 988 0.045) chunkcleanupworker stdout | 2023-08-08 06:18:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:18:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:22.293757+00:00 (in 9.999469 seconds) chunkcleanupworker stdout | 2023-08-08 06:18:12,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:22 UTC)" (scheduled at 2023-08-08 06:18:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:18:12,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:18:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 12, 295226), True, datetime.datetime(2023, 8, 8, 6, 18, 12, 295226), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:18:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:18:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:18:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:18:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:18:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:54.284093+00:00 (in 41.186222 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:18:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:13 UTC)" (scheduled at 2023-08-08 06:18:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:18:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:18:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 13, 98450), True, datetime.datetime(2023, 8, 8, 6, 18, 13, 98450), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:18:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:18:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:18:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:13 UTC)" executed successfully builder stdout | 2023-08-08 06:18:13,578 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:18:13,587 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:18:13,587 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:18:13,591 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:18:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:25.371682+00:00 (in 9.999502 seconds) notificationworker stdout | 2023-08-08 06:18:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:25 UTC)" (scheduled at 2023-08-08 06:18:15.371682+00:00) notificationworker stdout | 2023-08-08 06:18:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:18:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 15, 372558), True, datetime.datetime(2023, 8, 8, 6, 18, 15, 372558), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:18:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:18:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:18:15,501 [200] [DEBUG] [app] Starting request: urn:request:c2555b77-468b-4037-9736-6a0517fc9568 (/api/v1/signout) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:15,501 [200] [DEBUG] [app] User loader loading deferred user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:18:15,501 [200] [DEBUG] [auth.cookie] Loading user from cookie: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:18:15,502 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['fa311562-20d9-45f9-84c6-ffe37c4bf93c', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:15,508 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:15,508 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:15,508 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: fa311562-20d9-45f9-84c6-ffe37c4bf93c gunicorn-web stdout | 2023-08-08 06:18:15,509 [200] [DEBUG] [peewee] ('UPDATE "user" SET "uuid" = %s, "username" = %s, "password_hash" = %s, "email" = %s, "verified" = %s, "stripe_id" = %s, "organization" = %s, "robot" = %s, "invoice_email" = %s, "invalid_login_attempts" = %s, "last_invalid_login" = %s, "removed_tag_expiration_s" = %s, "enabled" = %s, "invoice_email_address" = %s, "given_name" = %s, "family_name" = %s, "company" = %s, "location" = %s, "maximum_queued_builds_count" = %s, "creation_date" = %s, "last_accessed" = %s WHERE ("user"."id" = %s)', ['f79914a3-7c73-4920-8b04-48f1ce35b138', 'whuquay', '$2b$12$.Zcr5ltcL08GZX2opR4Rz.7y4T10P3vewK0..b1skQUIzdrbJjh1u', 'quay+test@redhat.com', True, None, False, False, False, 0, datetime.datetime(2023, 8, 8, 5, 43, 24, 245581), 1209600, True, None, None, None, None, None, None, datetime.datetime(2023, 8, 8, 5, 43, 24, 245583), None, 1]) gunicorn-web stdout | 2023-08-08 06:18:15,513 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:15,513 [200] [ERROR] [auth.permissions] Unknown identity auth type: None gunicorn-web stdout | 2023-08-08 06:18:15,514 [200] [DEBUG] [app] Ending request: urn:request:c2555b77-468b-4037-9736-6a0517fc9568 (/api/v1/signout) {'endpoint': 'api.signout', 'request_id': 'urn:request:c2555b77-468b-4037-9736-6a0517fc9568', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/signout', 'path': '/api/v1/signout', 'parameters': {}, 'json_body': {}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:15,514 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:15 +0000] "POST /api/v1/signout HTTP/2.0" 200 18 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 419 0.015) gunicorn-web stdout | 2023-08-08 06:18:15,515 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:15 +0000] "POST /api/v1/signout HTTP/1.0" 200 18 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:15,836 [199] [DEBUG] [app] Starting request: urn:request:8c3717d7-d5f2-4034-85fc-8541050dd648 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:15,837 [199] [DEBUG] [app] Ending request: urn:request:8c3717d7-d5f2-4034-85fc-8541050dd648 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:8c3717d7-d5f2-4034-85fc-8541050dd648', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:15 +0000] "GET /api/v1/user/ HTTP/2.0" 401 306 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.003 348 0.004) gunicorn-web stdout | 2023-08-08 06:18:15,838 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:15 +0000] "GET /api/v1/user/ HTTP/1.0" 401 306 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/build_test/custom?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:16,132 [199] [DEBUG] [app] Starting request: urn:request:67d2874b-997c-4c21-8e5b-b594d550c305 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:16,132 [200] [DEBUG] [app] Starting request: urn:request:a3443776-46fe-4b93-aed7-422ea79f432b (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:16,132 [197] [DEBUG] [app] Starting request: urn:request:ef6e8759-7992-406e-a327-2b1a04292e65 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:16,132 [199] [DEBUG] [peewee] ('SELECT "t1"."id" AS "rid" FROM "repository" AS "t1" WHERE ("t1"."id" = %s)', [-1]) gunicorn-web stdout | 2023-08-08 06:18:16,133 [197] [DEBUG] [peewee] ('SELECT "t1"."id" AS "rid" FROM "repository" AS "t1" WHERE ("t1"."id" = %s)', [-1]) gunicorn-web stdout | 2023-08-08 06:18:16,133 [200] [DEBUG] [app] Ending request: urn:request:a3443776-46fe-4b93-aed7-422ea79f432b (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:a3443776-46fe-4b93-aed7-422ea79f432b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:16 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 400 92 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.004 165 0.003) gunicorn-web stdout | 2023-08-08 06:18:16,134 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:16 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 400 92 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:16,138 [199] [DEBUG] [app] Ending request: urn:request:67d2874b-997c-4c21-8e5b-b594d550c305 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:67d2874b-997c-4c21-8e5b-b594d550c305', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'whuquay', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:16,139 [197] [DEBUG] [app] Ending request: urn:request:ef6e8759-7992-406e-a327-2b1a04292e65 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:ef6e8759-7992-406e-a327-2b1a04292e65', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'false', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:16,139 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:16,139 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:16,139 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:16 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:16 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.009 103 0.009) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:16 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.010 114 0.009) gunicorn-web stdout | 2023-08-08 06:18:16,140 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:16 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=false"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:18:18,482 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:18:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:18:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:29.417660+00:00 (in 9.999467 seconds) gcworker stdout | 2023-08-08 06:18:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:29 UTC)" (scheduled at 2023-08-08 06:18:19.417660+00:00) gcworker stdout | 2023-08-08 06:18:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265899418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:18:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:18:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:18:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:18:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:18:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:26.890741+00:00 (in 7.202856 seconds) securityworker stdout | 2023-08-08 06:18:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:49 UTC)" (scheduled at 2023-08-08 06:18:19.687363+00:00) securityworker stdout | 2023-08-08 06:18:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:18:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:18:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:18:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:18:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:18:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:18:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 13, 19, 692164), 1, 2]) securityworker stdout | 2023-08-08 06:18:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 13, 19, 692164), 1, 2]) securityworker stdout | 2023-08-08 06:18:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:18:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:18:19,745 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:18:20,794 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:18:21,259 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:18:22,179 [200] [DEBUG] [app] Starting request: urn:request:9cc7c533-7eb8-4dbc-9df5-1e64c6610a87 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:18:22,179 [197] [DEBUG] [app] Starting request: urn:request:37583f08-8e4e-40ff-919f-ff40f4bff04f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:18:22,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:18:22,184 [210] [DEBUG] [app] Starting request: urn:request:52147e73-d602-45d1-858b-85e54024c659 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:18:22,185 [203] [DEBUG] [app] Starting request: urn:request:3259d5d1-56ee-4caf-88da-4afa799f147d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:18:22,185 [210] [DEBUG] [app] Ending request: urn:request:52147e73-d602-45d1-858b-85e54024c659 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:52147e73-d602-45d1-858b-85e54024c659', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:18:22,185 [203] [DEBUG] [app] Ending request: urn:request:3259d5d1-56ee-4caf-88da-4afa799f147d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3259d5d1-56ee-4caf-88da-4afa799f147d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:18:22,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:18:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:18:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:18:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:22,191 [200] [DEBUG] [app] Starting request: urn:request:4262f298-2c49-481a-8ea4-d4402e75c4df (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:18:22,191 [200] [DEBUG] [app] Ending request: urn:request:4262f298-2c49-481a-8ea4-d4402e75c4df (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4262f298-2c49-481a-8ea4-d4402e75c4df', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:18:22,191 [199] [DEBUG] [app] Starting request: urn:request:3a8ae281-c871-474b-b4e5-0a6e52b32122 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:18:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:18:22,192 [199] [DEBUG] [app] Ending request: urn:request:3a8ae281-c871-474b-b4e5-0a6e52b32122 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3a8ae281-c871-474b-b4e5-0a6e52b32122', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:18:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:18:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:18:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:18:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:18:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:18:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:18:22,201 [200] [DEBUG] [app] Ending request: urn:request:9cc7c533-7eb8-4dbc-9df5-1e64c6610a87 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9cc7c533-7eb8-4dbc-9df5-1e64c6610a87', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:18:22,201 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:22,202 [197] [DEBUG] [app] Ending request: urn:request:37583f08-8e4e-40ff-919f-ff40f4bff04f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:37583f08-8e4e-40ff-919f-ff40f4bff04f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:18:22,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:22,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:18:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:18:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:18:22,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:18:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:18:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) chunkcleanupworker stdout | 2023-08-08 06:18:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:18:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:28.874906+00:00 (in 6.580627 seconds) chunkcleanupworker stdout | 2023-08-08 06:18:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:32 UTC)" (scheduled at 2023-08-08 06:18:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:18:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:18:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 22, 294812), True, datetime.datetime(2023, 8, 8, 6, 18, 22, 294812), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:18:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:18:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:18:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:32 UTC)" executed successfully builder stdout | 2023-08-08 06:18:23,601 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:18:23,611 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:18:23,611 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:18:23,615 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:18:23,944 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:18:25,169 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:18:25,173 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:18:25,176 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:18:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:18:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:35.371682+00:00 (in 9.999511 seconds) notificationworker stdout | 2023-08-08 06:18:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:35 UTC)" (scheduled at 2023-08-08 06:18:25.371682+00:00) notificationworker stdout | 2023-08-08 06:18:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:18:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 25, 372678), True, datetime.datetime(2023, 8, 8, 6, 18, 25, 372678), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:18:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:18:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:18:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:18:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:33.514094+00:00 (in 7.999219 seconds) exportactionlogsworker stdout | 2023-08-08 06:18:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:25 UTC)" (scheduled at 2023-08-08 06:18:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:18:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:18:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:18:25,750 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:18:26,088 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:18:26,284 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:18:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:18:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:49.687363+00:00 (in 22.796090 seconds) securityworker stdout | 2023-08-08 06:18:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:56 UTC)" (scheduled at 2023-08-08 06:18:26.890741+00:00) securityworker stdout | 2023-08-08 06:18:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:18:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:18:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:18:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:18:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:18:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:18:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:18:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:18:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:18:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:18:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 13, 26, 896919), 1, 2]) securityworker stdout | 2023-08-08 06:18:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 13, 26, 896919), 1, 2]) securityworker stdout | 2023-08-08 06:18:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:18:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:18:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:18:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:18:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:18:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:18:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:18:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:18:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:18:27,213 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:18:27,575 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:18:27,700 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:18:27,759 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:18:27,793 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:18:27,795 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:18:27,877 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:18:27,892 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:18:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:18:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:32.293757+00:00 (in 3.418323 seconds) chunkcleanupworker stdout | 2023-08-08 06:18:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:28 UTC)" (scheduled at 2023-08-08 06:18:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:18:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:18:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:28 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:18:29,289 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} gcworker stdout | 2023-08-08 06:18:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:18:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:39.417660+00:00 (in 9.999462 seconds) gcworker stdout | 2023-08-08 06:18:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:39 UTC)" (scheduled at 2023-08-08 06:18:29.417660+00:00) gcworker stdout | 2023-08-08 06:18:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265909418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:18:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:18:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:18:29,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:39 UTC)" executed successfully builder stdout | 2023-08-08 06:18:29,432 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:18:30,488 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:18:31,352 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:18:31,475 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:18:31,920 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:18:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:18:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:42.293757+00:00 (in 9.999461 seconds) chunkcleanupworker stdout | 2023-08-08 06:18:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:42 UTC)" (scheduled at 2023-08-08 06:18:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:18:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:18:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 32, 294795), True, datetime.datetime(2023, 8, 8, 6, 18, 32, 294795), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:18:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:18:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:18:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:42 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:18:32,385 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:18:32,740 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-web stdout | 2023-08-08 06:18:33,009 [197] [DEBUG] [app] Starting request: urn:request:68f04481-db4c-46f1-b3bc-5f553bd3a02f (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:33,011 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ((("t1"."username" = %s) AND ("t1"."organization" = %s)) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,019 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['user1', 'bogus1@bogus.com', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,021 [197] [DEBUG] [data.model.user] Email and username are unique! gunicorn-web stdout | 2023-08-08 06:18:33,021 [197] [DEBUG] [peewee] ('INSERT INTO "user" ("uuid", "username", "email", "verified", "organization", "robot", "invoice_email", "invalid_login_attempts", "last_invalid_login", "removed_tag_expiration_s", "enabled", "maximum_queued_builds_count", "creation_date") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "user"."id"', ['506693df-aa05-419c-aa63-3bfe60e4edd1', 'user1', 'bogus1@bogus.com', False, False, False, False, 0, datetime.datetime(2023, 8, 8, 6, 18, 33, 21473), 1209600, True, None, datetime.datetime(2023, 8, 8, 6, 18, 33, 21474)]) expiredappspecifictokenworker stdout | 2023-08-08 06:18:33,082 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-web stdout | 2023-08-08 06:18:33,300 [197] [DEBUG] [peewee] ('UPDATE "user" SET "uuid" = %s, "username" = %s, "password_hash" = %s, "email" = %s, "verified" = %s, "organization" = %s, "robot" = %s, "invoice_email" = %s, "invalid_login_attempts" = %s, "last_invalid_login" = %s, "removed_tag_expiration_s" = %s, "enabled" = %s, "maximum_queued_builds_count" = %s, "creation_date" = %s WHERE ("user"."id" = %s)', ['506693df-aa05-419c-aa63-3bfe60e4edd1', 'user1', '$2b$12$2Sm8AjKaG.lwwtnVgToyA.svW9udtCMby4JKm3us0VpwnjnrrOWSK', 'bogus1@bogus.com', True, False, False, False, 0, datetime.datetime(2023, 8, 8, 6, 18, 33, 21473), 1209600, True, None, datetime.datetime(2023, 8, 8, 6, 18, 33, 21474), 3]) gunicorn-web stdout | 2023-08-08 06:18:33,306 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,308 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,309 [197] [DEBUG] [endpoints.common] Successfully signed in as user user1 with uuid 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,310 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:33,310 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,310 [197] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'user1']) gunicorn-web stdout | 2023-08-08 06:18:33,313 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,314 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,315 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:33,315 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:33,315 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:33,315 [197] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 3]) gunicorn-web stdout | 2023-08-08 06:18:33,317 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [3]) gunicorn-web stdout | 2023-08-08 06:18:33,318 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,320 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:18:33,322 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,323 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691475513322, 0, 3, None]) gunicorn-web stdout | 2023-08-08 06:18:33,326 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,328 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:18:33,329 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,330 [197] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691475513330, 0, 3]) gunicorn-web stdout | 2023-08-08 06:18:33,331 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,332 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,333 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,333 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,335 [197] [DEBUG] [app] Ending request: urn:request:68f04481-db4c-46f1-b3bc-5f553bd3a02f (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:68f04481-db4c-46f1-b3bc-5f553bd3a02f', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': {'password': '[FILTERED]', 'username': 'user1', 'email': 'bogus1@bogus.com', 'repeatPassword': '12345678'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:33,336 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:33,336 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:33 +0000] "POST /api/v1/user/ HTTP/1.0" 200 647 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:33 +0000] "POST /api/v1/user/ HTTP/2.0" 200 647 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.328 980 0.327) gunicorn-registry stdout | 2023-08-08 06:18:33,461 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:18:33,468 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:18:33,472 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:18:33,477 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:18:33,482 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:18:33,492 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:18:33,496 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:18:33,501 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-registry stdout | 2023-08-08 06:18:33,504 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} exportactionlogsworker stdout | 2023-08-08 06:18:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:18:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:47.398551+00:00 (in 13.883946 seconds) exportactionlogsworker stdout | 2023-08-08 06:18:33,514 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:23:33 UTC)" (scheduled at 2023-08-08 06:18:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:18:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 18, 33, 515062), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:18:33,521 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 18, 33, 515062), True, datetime.datetime(2023, 8, 8, 6, 18, 33, 515062), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:18:33,523 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 18, 33, 515062), True, datetime.datetime(2023, 8, 8, 6, 18, 33, 515062), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 18, 33, 515062), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:18:33,525 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:18:33,525 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:23:33 UTC)" executed successfully builder stdout | 2023-08-08 06:18:33,624 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:18:33,632 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:18:33,632 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:18:33,636 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:33,729 [197] [DEBUG] [app] Starting request: urn:request:8295ef73-4d4b-48e3-9c33-2ebf3e2e6aac (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:33,729 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,729 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,730 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,736 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:33,736 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:33,736 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,736 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,736 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:33,736 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:33,736 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:33,737 [197] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'user1']) gunicorn-web stdout | 2023-08-08 06:18:33,739 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,740 [197] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 3]) gunicorn-web stdout | 2023-08-08 06:18:33,741 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [3]) gunicorn-web stdout | 2023-08-08 06:18:33,742 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,744 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:18:33,745 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,747 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691475513746, 0, 3, None]) gunicorn-web stdout | 2023-08-08 06:18:33,750 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,751 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:18:33,752 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:33,753 [197] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691475513753, 0, 3]) gunicorn-web stdout | 2023-08-08 06:18:33,755 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,755 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:33,755 [197] [DEBUG] [app] Ending request: urn:request:8295ef73-4d4b-48e3-9c33-2ebf3e2e6aac (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:8295ef73-4d4b-48e3-9c33-2ebf3e2e6aac', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:33,756 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:33,757 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:33 +0000] "GET /api/v1/user/ HTTP/1.0" 200 645 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:33 +0000] "GET /api/v1/user/ HTTP/2.0" 200 645 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 442 0.029) gunicorn-web stdout | 2023-08-08 06:18:34,140 [200] [DEBUG] [app] Starting request: urn:request:4901b1ae-3d11-40f4-a1aa-c5cd7aa78b20 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:34,140 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,140 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,141 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:34,145 [198] [DEBUG] [app] Starting request: urn:request:dd24819f-7dbf-4bf8-87c9-8bd737f48761 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:34,145 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,145 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,146 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:34,147 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:34,147 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:34,147 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,147 [200] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:18:34,147 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,147 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,147 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,147 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,149 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:18:34,151 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:34,152 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:34,152 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,152 [198] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:18:34,152 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,152 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,152 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,152 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,153 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 30, 34, 152634), datetime.datetime(2023, 8, 8, 6, 18, 34, 152684)]) gunicorn-web stdout | 2023-08-08 06:18:34,154 [200] [DEBUG] [app] Ending request: urn:request:4901b1ae-3d11-40f4-a1aa-c5cd7aa78b20 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:4901b1ae-3d11-40f4-a1aa-c5cd7aa78b20', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:34,154 [198] [DEBUG] [app] Ending request: urn:request:dd24819f-7dbf-4bf8-87c9-8bd737f48761 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:dd24819f-7dbf-4bf8-87c9-8bd737f48761', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:34,154 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:34,155 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:34,155 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:34 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:34 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 127 0.015) gunicorn-web stdout | 2023-08-08 06:18:34,155 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:34 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:34 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.011 62 0.012) gunicorn-web stdout | 2023-08-08 06:18:34,439 [200] [DEBUG] [app] Starting request: urn:request:926f2870-3378-4e6f-a370-0e0a90d1283a (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:34,439 [197] [DEBUG] [app] Starting request: urn:request:55fdeed7-a049-4664-bc1e-946df3a260fb (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:34,439 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,439 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,440 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,440 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,440 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:34,440 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:34,447 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:34,447 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:34,447 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,447 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:34,447 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:34,447 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,447 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:34,447 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,447 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,447 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:34,448 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:18:34,448 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:34,450 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2023-08-08 06:18:34,452 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:18:34,452 [197] [DEBUG] [peewee] ('((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t3"."user_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'user1', 3, 3, 1, 'user1', 3, 3, 1, 'user1', 1, 3, 101]) gunicorn-web stdout | 2023-08-08 06:18:34,454 [200] [DEBUG] [app] Ending request: urn:request:926f2870-3378-4e6f-a370-0e0a90d1283a (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:926f2870-3378-4e6f-a370-0e0a90d1283a', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:34,454 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:34 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 84 0.016) gunicorn-web stdout | 2023-08-08 06:18:34,455 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:34 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:34,458 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:18:34,460 [197] [DEBUG] [app] Ending request: urn:request:55fdeed7-a049-4664-bc1e-946df3a260fb (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:55fdeed7-a049-4664-bc1e-946df3a260fb', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'user1', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:34,460 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:34 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.023 101 0.022) gunicorn-web stdout | 2023-08-08 06:18:34,461 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:34 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:18:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:18:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:45.371682+00:00 (in 9.999504 seconds) notificationworker stdout | 2023-08-08 06:18:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:45 UTC)" (scheduled at 2023-08-08 06:18:35.371682+00:00) notificationworker stdout | 2023-08-08 06:18:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:18:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 35, 372690), True, datetime.datetime(2023, 8, 8, 6, 18, 35, 372690), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:18:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:18:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:18:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:18:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:54.393627+00:00 (in 17.813247 seconds) namespacegcworker stdout | 2023-08-08 06:18:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:36 UTC)" (scheduled at 2023-08-08 06:18:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:18:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:18:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:18:37,178 [199] [DEBUG] [app] Starting request: urn:request:59e7f54b-156b-454a-aada-ba54218cdae1 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:18:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:37,180 [197] [DEBUG] [app] Starting request: urn:request:317e6042-a026-409c-b8c4-b99b4f8bc807 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:18:37,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:37,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:18:37,185 [203] [DEBUG] [app] Starting request: urn:request:a0b30c7f-0a89-4cb3-b9e7-2def0fdc1636 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:18:37,186 [203] [DEBUG] [app] Ending request: urn:request:a0b30c7f-0a89-4cb3-b9e7-2def0fdc1636 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a0b30c7f-0a89-4cb3-b9e7-2def0fdc1636', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:18:37,186 [204] [DEBUG] [app] Starting request: urn:request:5973f98d-d6e5-4049-b5f8-bfa08c1292d4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.004) gunicorn-registry stdout | 2023-08-08 06:18:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:18:37,187 [204] [DEBUG] [app] Ending request: urn:request:5973f98d-d6e5-4049-b5f8-bfa08c1292d4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5973f98d-d6e5-4049-b5f8-bfa08c1292d4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:18:37,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:18:37,187 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:18:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:37,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:37,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:37,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:37,192 [198] [DEBUG] [app] Starting request: urn:request:6e2b795c-d019-42e5-9fd6-7b84fff39e5e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:18:37,193 [200] [DEBUG] [app] Starting request: urn:request:086734cb-9e2c-4060-a989-7fc9db751bc8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:18:37,193 [198] [DEBUG] [app] Ending request: urn:request:6e2b795c-d019-42e5-9fd6-7b84fff39e5e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6e2b795c-d019-42e5-9fd6-7b84fff39e5e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:18:37,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:18:37,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:37,193 [200] [DEBUG] [app] Ending request: urn:request:086734cb-9e2c-4060-a989-7fc9db751bc8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:086734cb-9e2c-4060-a989-7fc9db751bc8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:18:37,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2023-08-08 06:18:37,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:18:37,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:37,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:18:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:18:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:07.197412+00:00 (in 29.999526 seconds) buildlogsarchiver stdout | 2023-08-08 06:18:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:07 UTC)" (scheduled at 2023-08-08 06:18:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:18:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 18, 37, 198307), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:37,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:18:37,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:18:37,204 [197] [DEBUG] [app] Ending request: urn:request:317e6042-a026-409c-b8c4-b99b4f8bc807 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:317e6042-a026-409c-b8c4-b99b4f8bc807', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:18:37,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:18:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:18:37,204 [199] [DEBUG] [app] Ending request: urn:request:59e7f54b-156b-454a-aada-ba54218cdae1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:59e7f54b-156b-454a-aada-ba54218cdae1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:18:37,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:37,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:18:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:18:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:18:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) buildlogsarchiver stdout | 2023-08-08 06:18:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:18:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:18:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:18:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:18:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:18:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:18:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:49.417660+00:00 (in 9.999463 seconds) gcworker stdout | 2023-08-08 06:18:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:49 UTC)" (scheduled at 2023-08-08 06:18:39.417660+00:00) gcworker stdout | 2023-08-08 06:18:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265919418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:18:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:18:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:18:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:18:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:18:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:52.293757+00:00 (in 9.999488 seconds) chunkcleanupworker stdout | 2023-08-08 06:18:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:52 UTC)" (scheduled at 2023-08-08 06:18:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:18:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:18:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 42, 294803), True, datetime.datetime(2023, 8, 8, 6, 18, 42, 294803), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:18:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:18:42,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:18:42,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:52 UTC)" executed successfully builder stdout | 2023-08-08 06:18:43,647 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:18:43,655 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:18:43,656 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:18:43,660 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:18:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:49.178248+00:00 (in 3.806056 seconds) notificationworker stdout | 2023-08-08 06:18:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:55 UTC)" (scheduled at 2023-08-08 06:18:45.371682+00:00) notificationworker stdout | 2023-08-08 06:18:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:18:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 45, 372690), True, datetime.datetime(2023, 8, 8, 6, 18, 45, 372690), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:18:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:18:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:55 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:18:45,904 [198] [DEBUG] [app] Starting request: urn:request:8a57429b-1bc1-4212-af7c-92a7779f52ab (/api/v1/organization/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:45,904 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:45,904 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:45,905 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:45,911 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:45,911 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:45,911 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:45,911 [198] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:18:45,912 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:45,912 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:45,912 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:45,912 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:45,913 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:45,915 [198] [DEBUG] [app] Ending request: urn:request:8a57429b-1bc1-4212-af7c-92a7779f52ab (/api/v1/organization/) {'endpoint': 'api.organizationlist', 'request_id': 'urn:request:8a57429b-1bc1-4212-af7c-92a7779f52ab', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/', 'path': '/api/v1/organization/', 'parameters': {}, 'json_body': {'name': 'build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:45,916 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:45 +0000] "POST /api/v1/organization/ HTTP/2.0" 400 343 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.014 946 0.013) gunicorn-web stdout | 2023-08-08 06:18:45,916 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:45 +0000] "POST /api/v1/organization/ HTTP/1.0" 400 343 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" exportactionlogsworker stdout | 2023-08-08 06:18:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:18:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:25.514341+00:00 (in 38.115263 seconds) exportactionlogsworker stdout | 2023-08-08 06:18:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:47 UTC)" (scheduled at 2023-08-08 06:18:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:18:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:18:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 47, 399740), True, datetime.datetime(2023, 8, 8, 6, 18, 47, 399740), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:18:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:18:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:18:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:18:48,496 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} notificationworker stdout | 2023-08-08 06:18:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:18:49,178 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:55.371682+00:00 (in 6.192881 seconds) notificationworker stdout | 2023-08-08 06:18:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:23:49 UTC)" (scheduled at 2023-08-08 06:18:49.178248+00:00) notificationworker stdout | 2023-08-08 06:18:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 18, 49, 179303), 'notification/%']) notificationworker stdout | 2023-08-08 06:18:49,186 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 18, 49, 179303), True, datetime.datetime(2023, 8, 8, 6, 18, 49, 179303), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:18:49,188 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 18, 49, 179303), True, datetime.datetime(2023, 8, 8, 6, 18, 49, 179303), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 18, 49, 179303), 'notification/%']) notificationworker stdout | 2023-08-08 06:18:49,190 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:49,190 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:23:49 UTC)" executed successfully gcworker stdout | 2023-08-08 06:18:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:18:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:59.417660+00:00 (in 9.999476 seconds) gcworker stdout | 2023-08-08 06:18:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:59 UTC)" (scheduled at 2023-08-08 06:18:49.417660+00:00) gcworker stdout | 2023-08-08 06:18:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265929418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:18:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:18:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:18:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:18:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:18:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:18:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:18:56.890741+00:00 (in 7.202848 seconds) securityworker stdout | 2023-08-08 06:18:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:19 UTC)" (scheduled at 2023-08-08 06:18:49.687363+00:00) securityworker stdout | 2023-08-08 06:18:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:18:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:18:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:18:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:18:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:18:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:18:49,778 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:18:50,832 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:18:51,292 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:18:52,179 [197] [DEBUG] [app] Starting request: urn:request:17ea3650-1c4e-46b0-a6f2-a00a1076b6a6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:18:52,179 [200] [DEBUG] [app] Starting request: urn:request:c29a3c7c-a675-4cd2-a2c5-a67680c62cc2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:18:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:18:52,184 [203] [DEBUG] [app] Starting request: urn:request:e9edc343-840e-4e29-be32-6d7e1ae81de7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:18:52,185 [210] [DEBUG] [app] Starting request: urn:request:f9671ada-b8bd-4970-af70-6849bfb26ff8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:18:52,185 [203] [DEBUG] [app] Ending request: urn:request:e9edc343-840e-4e29-be32-6d7e1ae81de7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e9edc343-840e-4e29-be32-6d7e1ae81de7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:18:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:18:52,185 [210] [DEBUG] [app] Ending request: urn:request:f9671ada-b8bd-4970-af70-6849bfb26ff8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f9671ada-b8bd-4970-af70-6849bfb26ff8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:18:52,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:18:52,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:18:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:18:52,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:52,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:18:52,191 [197] [DEBUG] [app] Starting request: urn:request:41b3145b-4a1c-4a25-82fd-394127fc5206 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:18:52,192 [197] [DEBUG] [app] Ending request: urn:request:41b3145b-4a1c-4a25-82fd-394127fc5206 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:41b3145b-4a1c-4a25-82fd-394127fc5206', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:18:52,192 [198] [DEBUG] [app] Starting request: urn:request:594520bf-3827-4436-bca3-f3472bde4339 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:18:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:18:52,192 [198] [DEBUG] [app] Ending request: urn:request:594520bf-3827-4436-bca3-f3472bde4339 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:594520bf-3827-4436-bca3-f3472bde4339', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:18:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:18:52,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:18:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:18:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:18:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:18:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:18:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:18:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:18:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:18:52,202 [197] [DEBUG] [app] Ending request: urn:request:17ea3650-1c4e-46b0-a6f2-a00a1076b6a6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:17ea3650-1c4e-46b0-a6f2-a00a1076b6a6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:18:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:52,203 [200] [DEBUG] [app] Ending request: urn:request:c29a3c7c-a675-4cd2-a2c5-a67680c62cc2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c29a3c7c-a675-4cd2-a2c5-a67680c62cc2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:18:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:18:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:18:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:18:52,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:18:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:18:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:18:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:18:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:18:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:02.293757+00:00 (in 9.999479 seconds) chunkcleanupworker stdout | 2023-08-08 06:18:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:02 UTC)" (scheduled at 2023-08-08 06:18:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:18:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:18:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 52, 294748), True, datetime.datetime(2023, 8, 8, 6, 18, 52, 294748), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:18:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:18:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:18:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:02 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:18:52,841 [198] [DEBUG] [app] Starting request: urn:request:cb7db8c0-2f46-4e5c-86a4-6b733f12694d (/api/v1/organization/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:52,842 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:52,842 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:52,842 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:52,849 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:52,849 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:52,849 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:52,850 [198] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:18:52,850 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:52,850 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:52,850 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:52,850 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:52,851 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:52,853 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:52,855 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', '318ed8c4-13d3-4b94-8949-8bc46eeba927', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:52,856 [198] [DEBUG] [data.model.user] Email and username are unique! gunicorn-web stdout | 2023-08-08 06:18:52,857 [198] [DEBUG] [peewee] ('INSERT INTO "user" ("uuid", "username", "email", "verified", "organization", "robot", "invoice_email", "invalid_login_attempts", "last_invalid_login", "removed_tag_expiration_s", "enabled", "maximum_queued_builds_count", "creation_date") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "user"."id"', ['2806f154-b4d5-4d3c-80a7-b1d1100628f1', 'user1_build_test', '318ed8c4-13d3-4b94-8949-8bc46eeba927', False, False, False, False, 0, datetime.datetime(2023, 8, 8, 6, 18, 52, 856861), 1209600, True, None, datetime.datetime(2023, 8, 8, 6, 18, 52, 856862)]) gunicorn-web stdout | 2023-08-08 06:18:52,858 [198] [DEBUG] [peewee] ('UPDATE "user" SET "uuid" = %s, "username" = %s, "email" = %s, "verified" = %s, "organization" = %s, "robot" = %s, "invoice_email" = %s, "invalid_login_attempts" = %s, "last_invalid_login" = %s, "removed_tag_expiration_s" = %s, "enabled" = %s, "maximum_queued_builds_count" = %s, "creation_date" = %s WHERE ("user"."id" = %s)', ['2806f154-b4d5-4d3c-80a7-b1d1100628f1', 'user1_build_test', '318ed8c4-13d3-4b94-8949-8bc46eeba927', False, True, False, False, 0, datetime.datetime(2023, 8, 8, 6, 18, 52, 856861), 1209600, True, None, datetime.datetime(2023, 8, 8, 6, 18, 52, 856862), 4]) gunicorn-web stdout | 2023-08-08 06:18:52,859 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:52,860 [198] [DEBUG] [peewee] ('INSERT INTO "team" ("name", "organization_id", "role_id", "description") VALUES (%s, %s, %s, %s) RETURNING "team"."id"', ['owners', 4, 1, '']) gunicorn-web stdout | 2023-08-08 06:18:52,861 [198] [DEBUG] [peewee] ('INSERT INTO "teammember" ("user_id", "team_id") VALUES (%s, %s) RETURNING "teammember"."id"', [3, 2]) gunicorn-web stdout | 2023-08-08 06:18:52,865 [198] [DEBUG] [app] Ending request: urn:request:cb7db8c0-2f46-4e5c-86a4-6b733f12694d (/api/v1/organization/) {'endpoint': 'api.organizationlist', 'request_id': 'urn:request:cb7db8c0-2f46-4e5c-86a4-6b733f12694d', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/', 'path': '/api/v1/organization/', 'parameters': {}, 'json_body': {'name': 'user1_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:52,866 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:52 +0000] "POST /api/v1/organization/ HTTP/2.0" 201 10 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 952 0.025) gunicorn-web stdout | 2023-08-08 06:18:52,866 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:52 +0000] "POST /api/v1/organization/ HTTP/1.0" 201 10 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:53,188 [197] [DEBUG] [app] Starting request: urn:request:adb287f9-da0d-46c6-b4b6-60894fdc2954 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:53,189 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,189 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,189 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,196 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:53,196 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:53,196 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,196 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,196 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,196 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,196 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,197 [197] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'user1']) gunicorn-web stdout | 2023-08-08 06:18:53,200 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,200 [197] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 3]) gunicorn-web stdout | 2023-08-08 06:18:53,202 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [3]) gunicorn-web stdout | 2023-08-08 06:18:53,203 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,204 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:18:53,206 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,208 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691475533207, 0, 3, None]) gunicorn-web stdout | 2023-08-08 06:18:53,211 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,211 [199] [DEBUG] [app] Starting request: urn:request:d4e0c13f-5328-483f-a463-048f9d8c7597 (/api/v1/organization/user1_build_test) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:53,211 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,212 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,212 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:18:53,212 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,213 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,215 [197] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691475533214, 0, 3]) gunicorn-web stdout | 2023-08-08 06:18:53,216 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,216 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,217 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:18:53,218 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:53,218 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:53,218 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,219 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,219 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,219 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,219 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,219 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,219 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,220 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,220 [197] [DEBUG] [app] Ending request: urn:request:adb287f9-da0d-46c6-b4b6-60894fdc2954 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:adb287f9-da0d-46c6-b4b6-60894fdc2954', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:53,220 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,220 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,220 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,221 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/user/ HTTP/2.0" 200 918 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.034 411 0.034) gunicorn-web stdout | 2023-08-08 06:18:53,221 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/user/ HTTP/1.0" 200 918 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organizations/new/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:53,221 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:18:53,223 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,224 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,224 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,224 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t2" ON ("t1"."role_id" = "t2"."id") WHERE ("t1"."organization_id" = %s)', [4]) gunicorn-web stdout | 2023-08-08 06:18:53,225 [199] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "repositorypermission" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [2]) gunicorn-web stdout | 2023-08-08 06:18:53,226 [199] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "teammember" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [2]) gunicorn-web stdout | 2023-08-08 06:18:53,227 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,227 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,227 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,228 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:18:53,229 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,231 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691475533230, 0, 4, None]) gunicorn-web stdout | 2023-08-08 06:18:53,233 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,235 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:18:53,236 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,238 [199] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691475533237, 0, 4]) gunicorn-web stdout | 2023-08-08 06:18:53,239 [199] [DEBUG] [app] Ending request: urn:request:d4e0c13f-5328-483f-a463-048f9d8c7597 (/api/v1/organization/user1_build_test) {'endpoint': 'api.organization', 'request_id': 'urn:request:d4e0c13f-5328-483f-a463-048f9d8c7597', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/user1_build_test', 'path': '/api/v1/organization/user1_build_test', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:53,240 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/organization/user1_build_test HTTP/2.0" 200 782 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.030 147 0.030) gunicorn-web stdout | 2023-08-08 06:18:53,240 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/organization/user1_build_test HTTP/1.0" 200 782 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:53,595 [197] [DEBUG] [app] Starting request: urn:request:ff7a1ad4-3fd0-428e-be61-49289f48621c (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:53,595 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,595 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,596 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,597 [199] [DEBUG] [app] Starting request: urn:request:71628bf2-855b-4c90-a63f-54cc9d8772c8 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:53,597 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,597 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,598 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,602 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:53,602 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:53,602 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,602 [197] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:18:53,602 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,602 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,602 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,602 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,604 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:53,604 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:53,604 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,604 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:18:53,604 [199] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:18:53,604 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,604 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,604 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,604 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,605 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 30, 53, 604919), datetime.datetime(2023, 8, 8, 6, 18, 53, 604970)]) gunicorn-web stdout | 2023-08-08 06:18:53,606 [199] [DEBUG] [app] Ending request: urn:request:71628bf2-855b-4c90-a63f-54cc9d8772c8 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:71628bf2-855b-4c90-a63f-54cc9d8772c8', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:53,607 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.011 62 0.012) gunicorn-web stdout | 2023-08-08 06:18:53,608 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:53,608 [197] [DEBUG] [app] Ending request: urn:request:ff7a1ad4-3fd0-428e-be61-49289f48621c (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:ff7a1ad4-3fd0-428e-be61-49289f48621c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:53,609 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 397 0.016) gunicorn-web stdout | 2023-08-08 06:18:53,609 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" builder stdout | 2023-08-08 06:18:53,670 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:18:53,678 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:18:53,678 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:18:53,682 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:53,718 [200] [DEBUG] [app] Starting request: urn:request:67d6bd68-cc3f-475a-9dc1-ab448038f994 (/api/v1/organization/user1_build_test/quota) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:53,718 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,718 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,718 [197] [DEBUG] [app] Starting request: urn:request:9022fe07-3b32-4cfc-8791-44f2bc33e07e (/api/v1/organization/user1_build_test/proxycache) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:53,719 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,719 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,719 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,719 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,724 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:53,725 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:53,725 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,725 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,725 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:53,725 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,725 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,725 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,725 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:53,725 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,725 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,725 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,725 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,725 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,726 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:18:53,727 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:18:53,729 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,729 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,729 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,729 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,729 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,729 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,729 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,729 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', True, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,731 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:18:53,731 [197] [DEBUG] [app] Ending request: urn:request:9022fe07-3b32-4cfc-8791-44f2bc33e07e (/api/v1/organization/user1_build_test/proxycache) {'endpoint': 'api.organizationproxycacheconfig', 'request_id': 'urn:request:9022fe07-3b32-4cfc-8791-44f2bc33e07e', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/user1_build_test/proxycache', 'path': '/api/v1/organization/user1_build_test/proxycache', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:53,732 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."quota_id", "t1"."quota_type_id", "t1"."percent_of_limit" FROM "quotalimits" AS "t1" INNER JOIN "quotatype" AS "t2" ON ("t1"."quota_type_id" = "t2"."id") WHERE ("t1"."quota_id" = %s)', [None]) gunicorn-web stdout | 2023-08-08 06:18:53,732 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:53,732 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/organization/user1_build_test/proxycache HTTP/1.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/organization/user1_build_test/proxycache HTTP/2.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.015 70 0.015) gunicorn-web stdout | 2023-08-08 06:18:53,733 [200] [DEBUG] [app] Ending request: urn:request:67d6bd68-cc3f-475a-9dc1-ab448038f994 (/api/v1/organization/user1_build_test/quota) {'endpoint': 'api.organizationquotalist', 'request_id': 'urn:request:67d6bd68-cc3f-475a-9dc1-ab448038f994', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/user1_build_test/quota', 'path': '/api/v1/organization/user1_build_test/quota', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:53,734 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:18:53,734 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/organization/user1_build_test/quota HTTP/1.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/organization/user1_build_test/quota HTTP/2.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 66 0.017) gunicorn-web stdout | 2023-08-08 06:18:53,856 [199] [DEBUG] [app] Starting request: urn:request:821f5624-3106-4909-8ce6-2d23ba207bd9 (/api/v1/organization/user1_build_test/prototypes) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:53,856 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,856 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,857 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,862 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:53,862 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:53,862 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,863 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,863 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,863 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,863 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,864 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:18:53,867 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,867 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,867 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,867 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,869 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" LEFT OUTER JOIN "user" AS "t2" ON ("t2"."id" = "t1"."activating_user_id") LEFT OUTER JOIN "user" AS "t3" ON ("t3"."id" = "t1"."delegate_user_id") LEFT OUTER JOIN "team" AS "t4" ON ("t4"."id" = "t1"."delegate_team_id") LEFT OUTER JOIN "role" AS "t5" ON ("t5"."id" = "t1"."role_id") WHERE ("t1"."org_id" = %s)', [4]) gunicorn-web stdout | 2023-08-08 06:18:53,871 [199] [DEBUG] [app] Ending request: urn:request:821f5624-3106-4909-8ce6-2d23ba207bd9 (/api/v1/organization/user1_build_test/prototypes) {'endpoint': 'api.permissionprototypelist', 'request_id': 'urn:request:821f5624-3106-4909-8ce6-2d23ba207bd9', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/user1_build_test/prototypes', 'path': '/api/v1/organization/user1_build_test/prototypes', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:53,872 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/organization/user1_build_test/prototypes HTTP/2.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 70 0.017) gunicorn-web stdout | 2023-08-08 06:18:53,872 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/organization/user1_build_test/prototypes HTTP/1.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:18:53,886 [199] [DEBUG] [app] Starting request: urn:request:9b9306f0-aa3a-4ea0-95fb-5b413c6f9517 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:18:53,886 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,886 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,886 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,892 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:18:53,892 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:18:53,892 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,892 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:18:53,892 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,892 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,892 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:18:53,893 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1', 1, 0]) gunicorn-web stdout | 2023-08-08 06:18:53,894 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2023-08-08 06:18:53,897 [199] [DEBUG] [peewee] ('(((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t1"."visibility_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t6"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'user1_build_test', 1, 3, 1, 'user1_build_test', 3, 3, 1, 'user1_build_test', 3, 3, 1, 'user1_build_test', 1, 3, 101]) gunicorn-web stdout | 2023-08-08 06:18:53,903 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:18:53,905 [199] [DEBUG] [app] Ending request: urn:request:9b9306f0-aa3a-4ea0-95fb-5b413c6f9517 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:9b9306f0-aa3a-4ea0-95fb-5b413c6f9517', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=true"a=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'user1_build_test', 'popularity': 'true', 'public': 'true', 'quota': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:18:53,905 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=true"a=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 110 0.021) gunicorn-web stdout | 2023-08-08 06:18:53,906 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:18:53 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=true"a=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gcworker stdout | 2023-08-08 06:18:53,977 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:18:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:18:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:13.097375+00:00 (in 18.812707 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:18:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:54 UTC)" (scheduled at 2023-08-08 06:18:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:18:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:18:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:18:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:18:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:36.579841+00:00 (in 42.185709 seconds) namespacegcworker stdout | 2023-08-08 06:18:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:54 UTC)" (scheduled at 2023-08-08 06:18:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:18:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:18:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 54, 394583), True, datetime.datetime(2023, 8, 8, 6, 18, 54, 394583), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:18:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:18:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:18:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:19:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:18:55,178 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:18:55,181 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:18:55,184 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:18:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:18:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:02.178530+00:00 (in 6.806348 seconds) notificationworker stdout | 2023-08-08 06:18:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:05 UTC)" (scheduled at 2023-08-08 06:18:55.371682+00:00) notificationworker stdout | 2023-08-08 06:18:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:18:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 18, 55, 372690), True, datetime.datetime(2023, 8, 8, 6, 18, 55, 372690), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:18:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:18:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:18:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:18:55,764 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:18:56,126 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:18:56,299 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:18:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:18:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:19.687363+00:00 (in 22.796119 seconds) securityworker stdout | 2023-08-08 06:18:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:26 UTC)" (scheduled at 2023-08-08 06:18:56.890741+00:00) securityworker stdout | 2023-08-08 06:18:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:18:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:18:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:18:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:18:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:18:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:18:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:18:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:18:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:18:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:18:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 13, 56, 896836), 1, 2]) securityworker stdout | 2023-08-08 06:18:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 13, 56, 896836), 1, 2]) securityworker stdout | 2023-08-08 06:18:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:18:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:18:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:18:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:18:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:18:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:18:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:18:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:18:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:18:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:18:57,250 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:18:57,583 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:18:57,713 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:18:57,773 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:18:57,807 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:18:57,810 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:18:57,891 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:18:57,929 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} buildlogsarchiver stdout | 2023-08-08 06:18:59,327 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} gcworker stdout | 2023-08-08 06:18:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:18:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:09.417660+00:00 (in 9.999485 seconds) gcworker stdout | 2023-08-08 06:18:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:09 UTC)" (scheduled at 2023-08-08 06:18:59.417660+00:00) gcworker stdout | 2023-08-08 06:18:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265939418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:18:59,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:18:59,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:18:59,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:09 UTC)" executed successfully builder stdout | 2023-08-08 06:18:59,450 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:19:00,521 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:19:01,389 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:19:01,513 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:19:01,957 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:19:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:19:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:05.371682+00:00 (in 3.192591 seconds) notificationworker stdout | 2023-08-08 06:19:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:02 UTC)" (scheduled at 2023-08-08 06:19:02.178530+00:00) notificationworker stdout | 2023-08-08 06:19:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:19:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:19:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:19:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:12.293757+00:00 (in 9.999477 seconds) chunkcleanupworker stdout | 2023-08-08 06:19:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:12 UTC)" (scheduled at 2023-08-08 06:19:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:19:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:19:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 2, 294767), True, datetime.datetime(2023, 8, 8, 6, 19, 2, 294767), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:19:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:19:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:19:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:12 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:19:02,423 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:19:02,748 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:19:03,090 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:19:03,474 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:19:03,480 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:19:03,484 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:19:03,487 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:19:03,490 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:19:03,501 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:19:03,508 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:19:03,512 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:19:03,515 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:19:03,692 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:19:03,701 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:19:03,701 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:19:03,705 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:19:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:19:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:09.793658+00:00 (in 5.605717 seconds) repositorygcworker stdout | 2023-08-08 06:19:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:04 UTC)" (scheduled at 2023-08-08 06:19:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:19:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:19:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:19:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:19:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:15.371682+00:00 (in 9.999492 seconds) notificationworker stdout | 2023-08-08 06:19:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:15 UTC)" (scheduled at 2023-08-08 06:19:05.371682+00:00) notificationworker stdout | 2023-08-08 06:19:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:19:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 5, 372701), True, datetime.datetime(2023, 8, 8, 6, 19, 5, 372701), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:19:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:19:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:19:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:07,179 [197] [DEBUG] [app] Starting request: urn:request:ccd87aa7-c7d9-48a3-9eb3-0e8d1f5a199d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:19:07,179 [200] [DEBUG] [app] Starting request: urn:request:9e0cfbef-3e38-4807-adc3-0e88c615f0da (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:19:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:19:07,185 [203] [DEBUG] [app] Starting request: urn:request:7fce3d07-021c-4bb2-a1e9-2971aee75efe (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:19:07,185 [209] [DEBUG] [app] Starting request: urn:request:b44374fd-1151-4987-830e-ae6c2def5985 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:19:07,185 [203] [DEBUG] [app] Ending request: urn:request:7fce3d07-021c-4bb2-a1e9-2971aee75efe (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7fce3d07-021c-4bb2-a1e9-2971aee75efe', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:19:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:19:07,186 [209] [DEBUG] [app] Ending request: urn:request:b44374fd-1151-4987-830e-ae6c2def5985 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b44374fd-1151-4987-830e-ae6c2def5985', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:19:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:19:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:19:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:07,191 [199] [DEBUG] [app] Starting request: urn:request:bca57655-720c-4a22-86d9-5235c21b6220 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:19:07,192 [199] [DEBUG] [app] Ending request: urn:request:bca57655-720c-4a22-86d9-5235c21b6220 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bca57655-720c-4a22-86d9-5235c21b6220', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:19:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:07,192 [198] [DEBUG] [app] Starting request: urn:request:4325626f-92ab-4dd8-acc7-c26fa39d8ff8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:19:07,193 [198] [DEBUG] [app] Ending request: urn:request:4325626f-92ab-4dd8-acc7-c26fa39d8ff8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4325626f-92ab-4dd8-acc7-c26fa39d8ff8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:19:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:19:07,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:19:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:19:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:19:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:19:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:37.197412+00:00 (in 29.999511 seconds) buildlogsarchiver stdout | 2023-08-08 06:19:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:37 UTC)" (scheduled at 2023-08-08 06:19:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:19:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 19, 7, 198460), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,202 [197] [DEBUG] [app] Ending request: urn:request:ccd87aa7-c7d9-48a3-9eb3-0e8d1f5a199d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ccd87aa7-c7d9-48a3-9eb3-0e8d1f5a199d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:19:07,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:19:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:19:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:19:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:19:07,203 [200] [DEBUG] [app] Ending request: urn:request:9e0cfbef-3e38-4807-adc3-0e88c615f0da (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9e0cfbef-3e38-4807-adc3-0e88c615f0da', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:19:07,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:19:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:19:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:19:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:19:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:19:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:19:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:19:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:19:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:37 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:07,504 [199] [DEBUG] [app] Starting request: urn:request:72430c61-ac67-407a-8617-6325cf7b1301 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:07,505 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,505 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,505 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,512 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:07,512 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:07,512 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,513 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,513 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,513 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,513 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,514 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:19:07,517 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,517 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,517 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,518 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,520 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,522 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,524 [199] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [4, 'github', 2, '', '547673b4-0f98-4bab-8b19-e06d0dac67eb', 1, False, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,526 [199] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [4, 0, datetime.date(2023, 8, 7)]) gunicorn-web stdout | 2023-08-08 06:19:07,527 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [4, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,527 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,528 [199] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [3, 4, 1]) gunicorn-web stdout | 2023-08-08 06:19:07,533 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_user_id" IS %s))', [4, 3, None, None]) gunicorn-web stdout | 2023-08-08 06:19:07,534 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_team_id" IS %s))', [4, 3, None, None]) gunicorn-web stdout | 2023-08-08 06:19:07,535 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,537 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,538 [199] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [14, 4, 3, 4, datetime.datetime(2023, 8, 8, 6, 19, 7, 537179), '10.130.2.1', '{"repo": "github", "namespace": "user1_build_test"}']) gunicorn-web stdout | 2023-08-08 06:19:07,541 [199] [DEBUG] [app] Ending request: urn:request:72430c61-ac67-407a-8617-6325cf7b1301 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:72430c61-ac67-407a-8617-6325cf7b1301', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository', 'path': '/api/v1/repository', 'parameters': {}, 'json_body': {'namespace': 'user1_build_test', 'repository': 'github', 'visibility': 'private', 'description': '', 'repo_kind': 'image'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:07,542 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:07 +0000] "POST /api/v1/repository HTTP/2.0" 201 69 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.039 1052 0.039) gunicorn-web stdout | 2023-08-08 06:19:07,542 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:07 +0000] "POST /api/v1/repository HTTP/1.0" 201 69 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:07,968 [197] [DEBUG] [app] Starting request: urn:request:6849c4c4-b3dc-44cd-88f3-0ada8fe4a99a (/api/v1/repository/user1_build_test/github/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:07,968 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,968 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,969 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,969 [200] [DEBUG] [app] Starting request: urn:request:f9a0a803-4296-427d-ab7a-fcc41851eb94 (/api/v1/repository/user1_build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:07,969 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,969 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,970 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,975 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:07,975 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:07,975 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,975 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:07,975 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:07,975 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:07,975 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,975 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,975 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,975 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,975 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,976 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:07,976 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,976 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,976 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,976 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,978 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:07,978 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:07,984 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,984 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:07,984 [200] [DEBUG] [endpoints.api.repository] Get repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:07,985 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,985 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,986 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [4, 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,988 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,988 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [4, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:19:07,989 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,989 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:07,990 [200] [DEBUG] [app] Ending request: urn:request:f9a0a803-4296-427d-ab7a-fcc41851eb94 (/api/v1/repository/user1_build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:f9a0a803-4296-427d-ab7a-fcc41851eb94', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false', 'path': '/api/v1/repository/user1_build_test/github', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:07,990 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [4, None, 1691475547989, False, 101, 0]) gunicorn-web stdout | 2023-08-08 06:19:07,991 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:07,991 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:07 +0000] "GET /api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false HTTP/1.0" 200 3632 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:07 +0000] "GET /api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false HTTP/2.0" 200 3632 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.024 92 0.024) gunicorn-web stdout | 2023-08-08 06:19:07,993 [197] [DEBUG] [app] Ending request: urn:request:6849c4c4-b3dc-44cd-88f3-0ada8fe4a99a (/api/v1/repository/user1_build_test/github/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:6849c4c4-b3dc-44cd-88f3-0ada8fe4a99a', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/user1_build_test/github/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:07,994 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:07 +0000] "GET /api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 544 0.029) gunicorn-web stdout | 2023-08-08 06:19:07,995 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:07 +0000] "GET /api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:08,674 [199] [DEBUG] [app] Starting request: urn:request:57f9e698-59cf-4d38-9242-90e372f57ecd (/repository/user1_build_test/github/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:08,675 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:08,683 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:08,685 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:08,688 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['github', 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:08,694 [199] [DEBUG] [app] Ending request: urn:request:57f9e698-59cf-4d38-9242-90e372f57ecd (/repository/user1_build_test/github/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:57f9e698-59cf-4d38-9242-90e372f57ecd', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/status?token=547673b4-0f98-4bab-8b19-e06d0dac67eb', 'path': '/repository/user1_build_test/github/status', 'parameters': {'token': '547673b4-0f98-4bab-8b19-e06d0dac67eb'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:08,694 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:08,695 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:08 +0000] "GET /repository/user1_build_test/github/status?token=547673b4-0f98-4bab-8b19-e06d0dac67eb HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:08 +0000] "GET /repository/user1_build_test/github/status?token=547673b4-0f98-4bab-8b19-e06d0dac67eb HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.022 129 0.022) gunicorn-web stdout | 2023-08-08 06:19:08,851 [197] [DEBUG] [app] Starting request: urn:request:e488b71a-d9d7-433a-b905-312a45b6deab (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:08,852 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:08,852 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:08,852 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:08,858 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:08,859 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:08,859 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:08,859 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:08,859 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:08,859 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:08,859 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:08,859 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:08,861 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:08,867 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:08,868 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:08,872 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:08,876 [197] [DEBUG] [app] Ending request: urn:request:e488b71a-d9d7-433a-b905-312a45b6deab (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:e488b71a-d9d7-433a-b905-312a45b6deab', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=3', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:08,876 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:08 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 77 0.027) gunicorn-web stdout | 2023-08-08 06:19:08,877 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:08 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gcworker stdout | 2023-08-08 06:19:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:19:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:19.417660+00:00 (in 9.999478 seconds) gcworker stdout | 2023-08-08 06:19:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:19 UTC)" (scheduled at 2023-08-08 06:19:09.417660+00:00) gcworker stdout | 2023-08-08 06:19:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265949418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:19:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:19:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:19:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:19:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:19:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:19.187230+00:00 (in 9.393045 seconds) repositorygcworker stdout | 2023-08-08 06:19:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:09 UTC)" (scheduled at 2023-08-08 06:19:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:19:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:19:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 9, 794562), True, datetime.datetime(2023, 8, 8, 6, 19, 9, 794562), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:19:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:19:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:19:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:09 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:10,201 [200] [DEBUG] [app] Starting request: urn:request:8de6b850-6648-4a14-b199-7cb7ff09190b (/api/v1/repository/user1_build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:10,201 [197] [DEBUG] [app] Starting request: urn:request:2f0617ff-a6d6-4180-8c52-c4f1b622cbe4 (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:10,202 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:10,202 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:10,202 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:10,202 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:10,202 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:10,202 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:10,209 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:10,209 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:10,209 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:10,209 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:10,209 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:10,209 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:10,209 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:10,209 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:10,209 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:10,209 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:10,209 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:10,209 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:10,209 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:10,209 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:10,209 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:10,209 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:10,212 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:10,212 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:10,218 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:10,218 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:10,219 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:10,219 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:10,222 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['user1_build_test', 'github']) gunicorn-web stdout | 2023-08-08 06:19:10,223 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 10]) gunicorn-web stdout | 2023-08-08 06:19:10,224 [200] [DEBUG] [app] Ending request: urn:request:8de6b850-6648-4a14-b199-7cb7ff09190b (/api/v1/repository/user1_build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:8de6b850-6648-4a14-b199-7cb7ff09190b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/', 'path': '/api/v1/repository/user1_build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:10,225 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:10 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/ HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.025 511 0.024) gunicorn-web stdout | 2023-08-08 06:19:10,225 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:10 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/ HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:10,226 [197] [DEBUG] [app] Ending request: urn:request:2f0617ff-a6d6-4180-8c52-c4f1b622cbe4 (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:2f0617ff-a6d6-4180-8c52-c4f1b622cbe4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=10', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:10,227 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:10 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 78 0.027) gunicorn-web stdout | 2023-08-08 06:19:10,227 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:10 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" chunkcleanupworker stdout | 2023-08-08 06:19:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:19:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:22.293757+00:00 (in 9.999459 seconds) chunkcleanupworker stdout | 2023-08-08 06:19:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:22 UTC)" (scheduled at 2023-08-08 06:19:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:19:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:19:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 12, 294820), True, datetime.datetime(2023, 8, 8, 6, 19, 12, 294820), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:19:12,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:19:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:19:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:19:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:19:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:54.284093+00:00 (in 41.186197 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:19:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:13 UTC)" (scheduled at 2023-08-08 06:19:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:19:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:19:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 13, 98368), True, datetime.datetime(2023, 8, 8, 6, 19, 13, 98368), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:19:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:19:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:19:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:13 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:13,141 [197] [DEBUG] [app] Starting request: urn:request:99b57907-d068-4690-b15c-c84f4662876d (/oauth2/github/callback/trigger/user1_build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:13,141 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:13,141 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:13,142 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:13,148 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:13,148 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:13,149 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:13,149 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:13,149 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:13,149 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:13,151 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:13,157 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:13,160 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443 gunicorn-web stdout | 2023-08-08 06:19:13,299 [197] [DEBUG] [urllib3.connectionpool] https://github.com:443 "POST /login/oauth/access_token?code=9cfa0a7c4def7de348bf&grant_type=authorization_code&redirect_uri=https%3A%2F%2Fquayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com%2Foauth2%2Fgithub%2Fcallback&client_id=2ddf9640aa6b5e341a7d&client_secret=3764a2acebe77c5406143cb606e79683e994ae38 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:13,301 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:13,304 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:13,305 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "buildtriggerservice" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['github', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:13,307 [197] [DEBUG] [peewee] ('INSERT INTO "repositorybuildtrigger" ("uuid", "service_id", "repository_id", "connected_user_id", "secure_auth_token", "fully_migrated", "config", "pull_robot_id", "enabled", "disabled_datetime", "successive_failure_count", "successive_internal_error_count") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repositorybuildtrigger"."id"', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 4, 3, 'v0$$O3aMXM8I1fqtTLpzWPYxULYnjaRrZDsbCG0gxW324/LHmRpcy4qsmB9/TIGS2uV//pjwWcIFKFpghI/qyrWNzz263aBy', False, '{}', None, True, datetime.datetime(2023, 8, 8, 6, 19, 13, 306528), 0, 0]) gunicorn-web stdout | 2023-08-08 06:19:13,312 [197] [DEBUG] [endpoints.githubtrigger] Redirecting to full url: /repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5 gunicorn-web stdout | 2023-08-08 06:19:13,313 [197] [DEBUG] [app] Ending request: urn:request:99b57907-d068-4690-b15c-c84f4662876d (/oauth2/github/callback/trigger/user1_build_test/github) {'endpoint': 'callback.attach_github_build_trigger', 'request_id': 'urn:request:99b57907-d068-4690-b15c-c84f4662876d', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/oauth2/github/callback/trigger/user1_build_test/github?code=9cfa0a7c4def7de348bf', 'path': '/oauth2/github/callback/trigger/user1_build_test/github', 'parameters': {'code': '9cfa0a7c4def7de348bf'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:13,314 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:13 +0000] "GET /oauth2/github/callback/trigger/user1_build_test/github?code=9cfa0a7c4def7de348bf HTTP/2.0" 302 367 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.174 629 0.174) gunicorn-web stdout | 2023-08-08 06:19:13,314 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:13 +0000] "GET /oauth2/github/callback/trigger/user1_build_test/github?code=9cfa0a7c4def7de348bf HTTP/1.0" 302 367 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:13,652 [199] [DEBUG] [app] Starting request: urn:request:599a0121-58a9-472b-9eb4-27628c41bfb3 (/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:13,653 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:13,656 [199] [DEBUG] [app] Ending request: urn:request:599a0121-58a9-472b-9eb4-27628c41bfb3 (/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5) {'endpoint': 'web.buildtrigger', 'request_id': 'urn:request:599a0121-58a9-472b-9eb4-27628c41bfb3', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 'path': '/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:13,656 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:13 +0000] "GET /repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5 HTTP/1.0" 200 85192 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:13 +0000] "GET /repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5 HTTP/2.0" 200 13315 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.006 571 0.006) builder stdout | 2023-08-08 06:19:13,715 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:19:13,724 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:19:13,724 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:19:13,728 [55] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:14 +0000] "GET /static/img/quay_favicon.png HTTP/2.0" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 181 -) gunicorn-web stdout | 2023-08-08 06:19:14,588 [200] [DEBUG] [app] Starting request: urn:request:dcbf80b5-729b-48ce-9d78-ad9ed62e236b (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:14,588 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,589 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,589 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,596 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:14,596 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:14,596 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,597 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,597 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,597 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,597 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,598 [200] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'user1']) gunicorn-web stdout | 2023-08-08 06:19:14,600 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,601 [200] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 3]) gunicorn-web stdout | 2023-08-08 06:19:14,602 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [3]) gunicorn-web stdout | 2023-08-08 06:19:14,603 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,604 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:19:14,606 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,607 [200] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691475554607, 0, 3, None]) gunicorn-web stdout | 2023-08-08 06:19:14,610 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,611 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:19:14,613 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,614 [200] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691475554613, 0, 3]) gunicorn-web stdout | 2023-08-08 06:19:14,615 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,615 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,616 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:19:14,618 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,618 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,618 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,619 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,619 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,619 [200] [DEBUG] [app] Ending request: urn:request:dcbf80b5-729b-48ce-9d78-ad9ed62e236b (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:dcbf80b5-729b-48ce-9d78-ad9ed62e236b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:14,620 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/user/ HTTP/2.0" 200 918 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.033 161 0.033) gunicorn-web stdout | 2023-08-08 06:19:14,620 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/user/ HTTP/1.0" 200 918 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:14,621 [197] [DEBUG] [app] Starting request: urn:request:20e53c27-2622-41cd-b5f7-187b5695fb41 (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:14,621 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,621 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,622 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,628 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:14,628 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:14,628 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,629 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 06:19:14,630 [197] [DEBUG] [app] Ending request: urn:request:20e53c27-2622-41cd-b5f7-187b5695fb41 (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:20e53c27-2622-41cd-b5f7-187b5695fb41', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:14,631 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.011 47 0.012) gunicorn-web stdout | 2023-08-08 06:19:14,631 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:14,692 [200] [DEBUG] [app] Starting request: urn:request:bfac13d8-c5f8-41f6-9d72-f91fe7263584 (/api/v1/repository/user1_build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:14,692 [197] [DEBUG] [app] Starting request: urn:request:5e84360b-848d-41d7-af9f-a2e9c60d68f6 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:14,693 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,693 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,693 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,693 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,693 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,693 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,699 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:14,699 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:14,699 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,699 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:14,700 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:14,700 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,700 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:14,700 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,700 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:14,700 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,700 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,700 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,700 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,700 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,700 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,700 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,702 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:14,702 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:14,708 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,708 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,709 [200] [DEBUG] [endpoints.api.repository] Get repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:14,710 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,710 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,711 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [4, 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,713 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [4, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:19:14,713 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,714 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,714 [200] [DEBUG] [app] Ending request: urn:request:bfac13d8-c5f8-41f6-9d72-f91fe7263584 (/api/v1/repository/user1_build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:bfac13d8-c5f8-41f6-9d72-f91fe7263584', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github?includeTags=false', 'path': '/api/v1/repository/user1_build_test/github', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:14,715 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,715 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:14,715 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/repository/user1_build_test/github?includeTags=false HTTP/1.0" 200 345 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/repository/user1_build_test/github?includeTags=false HTTP/2.0" 200 345 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.024 79 0.024) gunicorn-web stdout | 2023-08-08 06:19:14,718 [197] [DEBUG] [app] Ending request: urn:request:5e84360b-848d-41d7-af9f-a2e9c60d68f6 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:5e84360b-848d-41d7-af9f-a2e9c60d68f6', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:14,719 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5 HTTP/2.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 98 0.028) gunicorn-web stdout | 2023-08-08 06:19:14,720 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5 HTTP/1.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:14,932 [197] [DEBUG] [app] Starting request: urn:request:38ffc240-f883-437c-8cca-fee2b506727b (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:14,932 [200] [DEBUG] [app] Starting request: urn:request:19faed43-3d37-4ed0-9530-a2553a1bd300 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:14,932 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,932 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,932 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,932 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,933 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,933 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:14,939 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:14,939 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:14,939 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,939 [200] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:19:14,939 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,939 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,939 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,939 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:14,939 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,940 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:14,940 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,940 [197] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:19:14,940 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:14,940 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,940 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,940 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:14,941 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 31, 14, 940535), datetime.datetime(2023, 8, 8, 6, 19, 14, 940594)]) gunicorn-web stdout | 2023-08-08 06:19:14,941 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:19:14,942 [197] [DEBUG] [app] Ending request: urn:request:38ffc240-f883-437c-8cca-fee2b506727b (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:38ffc240-f883-437c-8cca-fee2b506727b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:14,943 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 62 0.013) gunicorn-web stdout | 2023-08-08 06:19:14,943 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:14,946 [200] [DEBUG] [app] Ending request: urn:request:19faed43-3d37-4ed0-9530-a2553a1bd300 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:19faed43-3d37-4ed0-9530-a2553a1bd300', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:14,946 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 475 0.016) gunicorn-web stdout | 2023-08-08 06:19:14,947 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:14 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:15,042 [197] [DEBUG] [app] Starting request: urn:request:59526267-8142-4722-bea2-2fb77009ee60 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/namespaces) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:15,042 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:15,042 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:15,043 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:15,049 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:15,049 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:15,049 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:15,049 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:15,050 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:15,050 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:15,050 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:15,050 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:15,052 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:15,058 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:15,059 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:15,062 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:15,066 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:15,066 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:15,068 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:15,196 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:15,197 [197] [DEBUG] [github.Requester] GET https://api.github.com/user {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:15 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"d246f06aab928037beb807b4945272ccdcccc342cfe09d6f864b736aff0f6650"', 'last-modified': 'Fri, 04 Aug 2023 13:02:06 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4941', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '59', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E26E:5801:EC6D33:1E4DE63:64D1DE63'} {"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false,"name":"Hu Weihua","company":null,"blog":"","location":null,"email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":15,"public_gists":0,"followers":5,"following":0,"created_at":"2014-04-22T08:41:09Z","updated_at":"2023-08-04T13:02:06Z"} gunicorn-web stdout | 2023-08-08 06:19:15,282 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user/orgs HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:15,283 [197] [DEBUG] [github.Requester] GET https://api.github.com/user/orgs {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:15 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"5998af94be686f343ddd78893fe706e1d53c16542fedc42ff2def6857f751c85"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:org, read:org, repo, user, write:org', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4940', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '60', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E26E:5801:EC6D4D:1E4DE91:64D1DE63'} [{"login":"finley","id":422903,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQyMjkwMw==","url":"https://api.github.com/orgs/finley","repos_url":"https://api.github.com/orgs/finley/repos","events_url":"https://api.github.com/orgs/finley/events","hooks_url":"https://api.github.com/orgs/finley/hooks","issues_url":"https://api.github.com/orgs/finley/issues","members_url":"https://api.github.com/orgs/finley/members{/member}","public_members_url":"https://api.github.com/orgs/finley/public_members{/member}","avatar_url":"https://avatars.githubusercontent.com/u/422903?v=4","description":null},{"login":"xcat2","id":10124414,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwMTI0NDE0","url":"https://api.github.com/orgs/xcat2","repos_url":"https://api.github.com/orgs/xcat2/repos","events_url":"https://api.github.com/orgs/xcat2/events","hooks_url":"https://api.github.com/orgs/xcat2/hooks","issues_url":"https://api.github.com/orgs/xcat2/issues","members_url":"https://api.github.com/orgs/xcat2/members{/member}","public_members_url":"https://api.github.com/orgs/xcat2/public_members{/member}","avatar_url":"https://avatars.githubusercontent.com/u/10124414?v=4","description":" xCAT offers complete management for HPC clusters, RenderFarms, Grids, WebFarms, Online Gaming Infrastructure, Clouds, Datacenters"}] gunicorn-web stdout | 2023-08-08 06:19:15,286 [197] [DEBUG] [app] Ending request: urn:request:59526267-8142-4722-bea2-2fb77009ee60 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/namespaces) {'endpoint': 'api.buildtriggersourcenamespaces', 'request_id': 'urn:request:59526267-8142-4722-bea2-2fb77009ee60', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/namespaces', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/namespaces', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:15,287 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:15 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/namespaces HTTP/2.0" 200 492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.247 106 0.246) gunicorn-web stdout | 2023-08-08 06:19:15,287 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:15 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/namespaces HTTP/1.0" 200 492 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:19:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:19:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:25.371682+00:00 (in 9.999466 seconds) notificationworker stdout | 2023-08-08 06:19:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:25 UTC)" (scheduled at 2023-08-08 06:19:15.371682+00:00) notificationworker stdout | 2023-08-08 06:19:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:19:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 15, 372713), True, datetime.datetime(2023, 8, 8, 6, 19, 15, 372713), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:19:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:19:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:19:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:17,210 [199] [DEBUG] [app] Starting request: urn:request:1e5e0911-6efa-4520-b0bb-aaf54a42d7ec (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/sources) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:17,210 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:17,210 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:17,211 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:17,217 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:17,217 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:17,217 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:17,217 [199] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:17,218 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:17,218 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:17,218 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:17,218 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:17,220 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:17,226 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:17,227 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:17,230 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:17,232 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:17,235 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:17,236 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:17,238 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:17,354 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:17,355 [199] [DEBUG] [github.Requester] GET https://api.github.com/user {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:17 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"d246f06aab928037beb807b4945272ccdcccc342cfe09d6f864b736aff0f6650"', 'last-modified': 'Fri, 04 Aug 2023 13:02:06 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4939', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '61', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E27E:6710:109C464:21F10C1:64D1DE65'} {"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false,"name":"Hu Weihua","company":null,"blog":"","location":null,"email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":15,"public_gists":0,"followers":5,"following":0,"created_at":"2014-04-22T08:41:09Z","updated_at":"2023-08-04T13:02:06Z"} gunicorn-web stdout | 2023-08-08 06:19:17,703 [199] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /user/repos?type=owner&sort=updated HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:17,704 [199] [DEBUG] [github.Requester] GET https://api.github.com/user/repos?type=owner&sort=updated {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:17 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"8bf877e6a612d7a65b26cda161a17c396587c86d52d213657fbc81f975431b7d"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4938', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '62', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E27E:6710:109C490:21F10FA:64D1DE65'} [{"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":473030298,"node_id":"R_kgDOHDHemg","name":"quay_build_test_repo2","full_name":"hu-weihua/quay_build_test_repo2","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test_repo2","description":"quay_build_test_repo2","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test_repo2/deployments","created_at":"2022-03-23T04:00:22Z","updated_at":"2022-03-23T04:01:50Z","pushed_at":"2022-03-23T04:01:47Z","git_url":"git://github.com/hu-weihua/quay_build_test_repo2.git","ssh_url":"git@github.com:hu-weihua/quay_build_test_repo2.git","clone_url":"https://github.com/hu-weihua/quay_build_test_repo2.git","svn_url":"https://github.com/hu-weihua/quay_build_test_repo2","homepage":null,"size":1,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":473864393,"node_id":"R_kgDOHD6YyQ","name":"node-app","full_name":"hu-weihua/node-app","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/node-app","description":null,"fork":true,"url":"https://api.github.com/repos/hu-weihua/node-app","forks_url":"https://api.github.com/repos/hu-weihua/node-app/forks","keys_url":"https://api.github.com/repos/hu-weihua/node-app/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/node-app/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/node-app/teams","hooks_url":"https://api.github.com/repos/hu-weihua/node-app/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/node-app/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/node-app/events","assignees_url":"https://api.github.com/repos/hu-weihua/node-app/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/node-app/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/node-app/tags","blobs_url":"https://api.github.com/repos/hu-weihua/node-app/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/node-app/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/node-app/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/node-app/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/node-app/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/node-app/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/node-app/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/node-app/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/node-app/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/node-app/subscription","commits_url":"https://api.github.com/repos/hu-weihua/node-app/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/node-app/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/node-app/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/node-app/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/node-app/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/node-app/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/node-app/merges","archive_url":"https://api.github.com/repos/hu-weihua/node-app/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/node-app/downloads","issues_url":"https://api.github.com/repos/hu-weihua/node-app/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/node-app/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/node-app/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/node-app/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/node-app/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/node-app/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/node-app/deployments","created_at":"2022-03-25T04:10:15Z","updated_at":"2021-12-30T22:39:01Z","pushed_at":"2022-03-11T17:49:16Z","git_url":"git://github.com/hu-weihua/node-app.git","ssh_url":"git@github.com:hu-weihua/node-app.git","clone_url":"https://github.com/hu-weihua/node-app.git","svn_url":"https://github.com/hu-weihua/node-app","homepage":null,"size":82,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":140396693,"node_id":"MDEwOlJlcG9zaXRvcnkxNDAzOTY2OTM=","name":"personal_repo","full_name":"hu-weihua/personal_repo","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/personal_repo","description":"This is repository of hwh","fork":false,"url":"https://api.github.com/repos/hu-weihua/personal_repo","forks_url":"https://api.github.com/repos/hu-weihua/personal_repo/forks","keys_url":"https://api.github.com/repos/hu-weihua/personal_repo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/personal_repo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/personal_repo/teams","hooks_url":"https://api.github.com/repos/hu-weihua/personal_repo/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/personal_repo/events","assignees_url":"https://api.github.com/repos/hu-weihua/personal_repo/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/personal_repo/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/personal_repo/tags","blobs_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/personal_repo/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/personal_repo/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/personal_repo/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/personal_repo/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/personal_repo/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/personal_repo/subscription","commits_url":"https://api.github.com/repos/hu-weihua/personal_repo/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/personal_repo/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/personal_repo/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/personal_repo/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/personal_repo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/personal_repo/merges","archive_url":"https://api.github.com/repos/hu-weihua/personal_repo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/personal_repo/downloads","issues_url":"https://api.github.com/repos/hu-weihua/personal_repo/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/personal_repo/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/personal_repo/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/personal_repo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/personal_repo/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/personal_repo/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/personal_repo/deployments","created_at":"2018-07-10T07:49:54Z","updated_at":"2021-11-04T03:05:14Z","pushed_at":"2021-11-04T03:05:11Z","git_url":"git://github.com/hu-weihua/personal_repo.git","ssh_url":"git@github.com:hu-weihua/personal_repo.git","clone_url":"https://github.com/hu-weihua/personal_repo.git","svn_url":"https://github.com/hu-weihua/personal_repo","homepage":null,"size":45089,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":411522280,"node_id":"R_kgDOGIdU6A","name":"quay_tools","full_name":"hu-weihua/quay_tools","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_tools","description":"scripts for quay ","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_tools","forks_url":"https://api.github.com/repos/hu-weihua/quay_tools/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_tools/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_tools/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_tools/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_tools/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_tools/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_tools/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_tools/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_tools/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_tools/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_tools/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_tools/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_tools/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_tools/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_tools/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_tools/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_tools/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_tools/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_tools/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_tools/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_tools/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_tools/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_tools/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_tools/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_tools/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_tools/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_tools/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_tools/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_tools/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_tools/deployments","created_at":"2021-09-29T03:52:46Z","updated_at":"2021-10-14T04:53:01Z","pushed_at":"2021-10-14T04:52:58Z","git_url":"git://github.com/hu-weihua/quay_tools.git","ssh_url":"git@github.com:hu-weihua/quay_tools.git","clone_url":"https://github.com/hu-weihua/quay_tools.git","svn_url":"https://github.com/hu-weihua/quay_tools","homepage":null,"size":5,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":409051481,"node_id":"R_kgDOGGGhWQ","name":"quay_fedaro_laptop_setup","full_name":"hu-weihua/quay_fedaro_laptop_setup","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup","forks_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_fedaro_laptop_setup/deployments","created_at":"2021-09-22T03:25:19Z","updated_at":"2021-09-22T03:32:29Z","pushed_at":"2021-09-22T03:32:26Z","git_url":"git://github.com/hu-weihua/quay_fedaro_laptop_setup.git","ssh_url":"git@github.com:hu-weihua/quay_fedaro_laptop_setup.git","clone_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup.git","svn_url":"https://github.com/hu-weihua/quay_fedaro_laptop_setup","homepage":null,"size":2,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":251479333,"node_id":"MDEwOlJlcG9zaXRvcnkyNTE0NzkzMzM=","name":"cam_tools","full_name":"hu-weihua/cam_tools","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/cam_tools","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/cam_tools","forks_url":"https://api.github.com/repos/hu-weihua/cam_tools/forks","keys_url":"https://api.github.com/repos/hu-weihua/cam_tools/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/cam_tools/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/cam_tools/teams","hooks_url":"https://api.github.com/repos/hu-weihua/cam_tools/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/cam_tools/events","assignees_url":"https://api.github.com/repos/hu-weihua/cam_tools/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/cam_tools/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/cam_tools/tags","blobs_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/cam_tools/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/cam_tools/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/cam_tools/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/cam_tools/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/cam_tools/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/cam_tools/subscription","commits_url":"https://api.github.com/repos/hu-weihua/cam_tools/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/cam_tools/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/cam_tools/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/cam_tools/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/cam_tools/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/cam_tools/merges","archive_url":"https://api.github.com/repos/hu-weihua/cam_tools/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/cam_tools/downloads","issues_url":"https://api.github.com/repos/hu-weihua/cam_tools/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/cam_tools/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/cam_tools/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/cam_tools/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/cam_tools/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/cam_tools/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/cam_tools/deployments","created_at":"2020-03-31T02:22:26Z","updated_at":"2021-07-12T10:24:29Z","pushed_at":"2021-07-12T10:24:26Z","git_url":"git://github.com/hu-weihua/cam_tools.git","ssh_url":"git@github.com:hu-weihua/cam_tools.git","clone_url":"https://github.com/hu-weihua/cam_tools.git","svn_url":"https://github.com/hu-weihua/cam_tools","homepage":null,"size":151,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":265992712,"node_id":"MDEwOlJlcG9zaXRvcnkyNjU5OTI3MTI=","name":"englist_study","full_name":"hu-weihua/englist_study","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/englist_study","description":"this is repo for english study","fork":false,"url":"https://api.github.com/repos/hu-weihua/englist_study","forks_url":"https://api.github.com/repos/hu-weihua/englist_study/forks","keys_url":"https://api.github.com/repos/hu-weihua/englist_study/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/englist_study/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/englist_study/teams","hooks_url":"https://api.github.com/repos/hu-weihua/englist_study/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/englist_study/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/englist_study/events","assignees_url":"https://api.github.com/repos/hu-weihua/englist_study/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/englist_study/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/englist_study/tags","blobs_url":"https://api.github.com/repos/hu-weihua/englist_study/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/englist_study/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/englist_study/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/englist_study/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/englist_study/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/englist_study/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/englist_study/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/englist_study/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/englist_study/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/englist_study/subscription","commits_url":"https://api.github.com/repos/hu-weihua/englist_study/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/englist_study/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/englist_study/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/englist_study/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/englist_study/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/englist_study/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/englist_study/merges","archive_url":"https://api.github.com/repos/hu-weihua/englist_study/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/englist_study/downloads","issues_url":"https://api.github.com/repos/hu-weihua/englist_study/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/englist_study/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/englist_study/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/englist_study/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/englist_study/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/englist_study/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/englist_study/deployments","created_at":"2020-05-22T01:45:08Z","updated_at":"2021-05-01T00:58:19Z","pushed_at":"2021-05-01T00:58:17Z","git_url":"git://github.com/hu-weihua/englist_study.git","ssh_url":"git@github.com:hu-weihua/englist_study.git","clone_url":"https://github.com/hu-weihua/englist_study.git","svn_url":"https://github.com/hu-weihua/englist_study","homepage":null,"size":82,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":308238023,"node_id":"MDEwOlJlcG9zaXRvcnkzMDgyMzgwMjM=","name":"rh-do380-ge","full_name":"hu-weihua/rh-do380-ge","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/rh-do380-ge","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/rh-do380-ge","forks_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/forks","keys_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/teams","hooks_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/events","assignees_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/tags","blobs_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/subscription","commits_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/merges","archive_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/downloads","issues_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/rh-do380-ge/deployments","created_at":"2020-10-29T06:30:04Z","updated_at":"2020-11-06T07:14:37Z","pushed_at":"2020-11-06T07:14:35Z","git_url":"git://github.com/hu-weihua/rh-do380-ge.git","ssh_url":"git@github.com:hu-weihua/rh-do380-ge.git","clone_url":"https://github.com/hu-weihua/rh-do380-ge.git","svn_url":"https://github.com/hu-weihua/rh-do380-ge","homepage":null,"size":8389,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":309275165,"node_id":"MDEwOlJlcG9zaXRvcnkzMDkyNzUxNjU=","name":"baisc_test_data","full_name":"hu-weihua/baisc_test_data","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/baisc_test_data","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/baisc_test_data","forks_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/forks","keys_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/teams","hooks_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/events","assignees_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/tags","blobs_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/subscription","commits_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/merges","archive_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/downloads","issues_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/baisc_test_data/deployments","created_at":"2020-11-02T06:12:55Z","updated_at":"2020-11-02T07:45:40Z","pushed_at":"2020-11-02T07:45:38Z","git_url":"git://github.com/hu-weihua/baisc_test_data.git","ssh_url":"git@github.com:hu-weihua/baisc_test_data.git","clone_url":"https://github.com/hu-weihua/baisc_test_data.git","svn_url":"https://github.com/hu-weihua/baisc_test_data","homepage":null,"size":7,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":308519553,"node_id":"MDEwOlJlcG9zaXRvcnkzMDg1MTk1NTM=","name":"gitops-gitops","full_name":"hu-weihua/gitops-gitops","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/gitops-gitops","description":null,"fork":false,"url":"https://api.github.com/repos/hu-weihua/gitops-gitops","forks_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/forks","keys_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/teams","hooks_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/events","assignees_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/tags","blobs_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/subscription","commits_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/merges","archive_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/downloads","issues_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/gitops-gitops/deployments","created_at":"2020-10-30T03:53:46Z","updated_at":"2020-10-30T04:27:19Z","pushed_at":"2020-10-30T04:27:16Z","git_url":"git://github.com/hu-weihua/gitops-gitops.git","ssh_url":"git@github.com:hu-weihua/gitops-gitops.git","clone_url":"https://github.com/hu-weihua/gitops-gitops.git","svn_url":"https://github.com/hu-weihua/gitops-gitops","homepage":null,"size":4,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":279471706,"node_id":"MDEwOlJlcG9zaXRvcnkyNzk0NzE3MDY=","name":"DO180-apps","full_name":"hu-weihua/DO180-apps","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/DO180-apps","description":"DO180 Repository for Sample Applications","fork":true,"url":"https://api.github.com/repos/hu-weihua/DO180-apps","forks_url":"https://api.github.com/repos/hu-weihua/DO180-apps/forks","keys_url":"https://api.github.com/repos/hu-weihua/DO180-apps/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/DO180-apps/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/DO180-apps/teams","hooks_url":"https://api.github.com/repos/hu-weihua/DO180-apps/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/DO180-apps/events","assignees_url":"https://api.github.com/repos/hu-weihua/DO180-apps/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/DO180-apps/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/DO180-apps/tags","blobs_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/DO180-apps/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/DO180-apps/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/DO180-apps/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/DO180-apps/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/DO180-apps/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/DO180-apps/subscription","commits_url":"https://api.github.com/repos/hu-weihua/DO180-apps/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/DO180-apps/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/DO180-apps/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/DO180-apps/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/DO180-apps/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/DO180-apps/merges","archive_url":"https://api.github.com/repos/hu-weihua/DO180-apps/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/DO180-apps/downloads","issues_url":"https://api.github.com/repos/hu-weihua/DO180-apps/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/DO180-apps/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/DO180-apps/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/DO180-apps/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/DO180-apps/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/DO180-apps/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/DO180-apps/deployments","created_at":"2020-07-14T03:29:50Z","updated_at":"2020-07-14T03:29:51Z","pushed_at":"2020-07-17T08:41:06Z","git_url":"git://github.com/hu-weihua/DO180-apps.git","ssh_url":"git@github.com:hu-weihua/DO180-apps.git","clone_url":"https://github.com/hu-weihua/DO180-apps.git","svn_url":"https://github.com/hu-weihua/DO180-apps","homepage":null,"size":230,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":238137702,"node_id":"MDEwOlJlcG9zaXRvcnkyMzgxMzc3MDI=","name":"mig-e2e","full_name":"hu-weihua/mig-e2e","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/mig-e2e","description":"End to end tests for OCP Migration","fork":true,"url":"https://a gunicorn-web stdout | pi.github.com/repos/hu-weihua/mig-e2e","forks_url":"https://api.github.com/repos/hu-weihua/mig-e2e/forks","keys_url":"https://api.github.com/repos/hu-weihua/mig-e2e/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/mig-e2e/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/mig-e2e/teams","hooks_url":"https://api.github.com/repos/hu-weihua/mig-e2e/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/mig-e2e/events","assignees_url":"https://api.github.com/repos/hu-weihua/mig-e2e/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/mig-e2e/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/mig-e2e/tags","blobs_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/mig-e2e/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/mig-e2e/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/mig-e2e/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/mig-e2e/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/mig-e2e/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/mig-e2e/subscription","commits_url":"https://api.github.com/repos/hu-weihua/mig-e2e/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/mig-e2e/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/mig-e2e/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/mig-e2e/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/mig-e2e/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/mig-e2e/merges","archive_url":"https://api.github.com/repos/hu-weihua/mig-e2e/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/mig-e2e/downloads","issues_url":"https://api.github.com/repos/hu-weihua/mig-e2e/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/mig-e2e/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/mig-e2e/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/mig-e2e/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/mig-e2e/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/mig-e2e/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/mig-e2e/deployments","created_at":"2020-02-04T06:21:19Z","updated_at":"2020-02-25T17:18:34Z","pushed_at":"2020-04-28T01:45:39Z","git_url":"git://github.com/hu-weihua/mig-e2e.git","ssh_url":"git@github.com:hu-weihua/mig-e2e.git","clone_url":"https://github.com/hu-weihua/mig-e2e.git","svn_url":"https://github.com/hu-weihua/mig-e2e","homepage":null,"size":211,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":228530382,"node_id":"MDEwOlJlcG9zaXRvcnkyMjg1MzAzODI=","name":"mig-operator","full_name":"hu-weihua/mig-operator","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/mig-operator","description":"OpenShift Migration Operator","fork":true,"url":"https://api.github.com/repos/hu-weihua/mig-operator","forks_url":"https://api.github.com/repos/hu-weihua/mig-operator/forks","keys_url":"https://api.github.com/repos/hu-weihua/mig-operator/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/mig-operator/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/mig-operator/teams","hooks_url":"https://api.github.com/repos/hu-weihua/mig-operator/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/mig-operator/events","assignees_url":"https://api.github.com/repos/hu-weihua/mig-operator/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/mig-operator/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/mig-operator/tags","blobs_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/mig-operator/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/mig-operator/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/mig-operator/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/mig-operator/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/mig-operator/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/mig-operator/subscription","commits_url":"https://api.github.com/repos/hu-weihua/mig-operator/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/mig-operator/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/mig-operator/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/mig-operator/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/mig-operator/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/mig-operator/merges","archive_url":"https://api.github.com/repos/hu-weihua/mig-operator/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/mig-operator/downloads","issues_url":"https://api.github.com/repos/hu-weihua/mig-operator/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/mig-operator/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/mig-operator/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/mig-operator/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/mig-operator/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/mig-operator/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/mig-operator/deployments","created_at":"2019-12-17T04:09:11Z","updated_at":"2020-02-25T14:54:53Z","pushed_at":"2019-12-16T22:52:14Z","git_url":"git://github.com/hu-weihua/mig-operator.git","ssh_url":"git@github.com:hu-weihua/mig-operator.git","clone_url":"https://github.com/hu-weihua/mig-operator.git","svn_url":"https://github.com/hu-weihua/mig-operator","homepage":"","size":15667,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":235289269,"node_id":"MDEwOlJlcG9zaXRvcnkyMzUyODkyNjk=","name":"cam_test","full_name":"hu-weihua/cam_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/cam_test","description":"just for cam test","fork":false,"url":"https://api.github.com/repos/hu-weihua/cam_test","forks_url":"https://api.github.com/repos/hu-weihua/cam_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/cam_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/cam_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/cam_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/cam_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/cam_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/cam_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/cam_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/cam_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/cam_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/cam_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/cam_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/cam_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/cam_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/cam_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/cam_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/cam_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/cam_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/cam_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/cam_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/cam_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/cam_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/cam_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/cam_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/cam_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/cam_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/cam_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/cam_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/cam_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/cam_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/cam_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/cam_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/cam_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/cam_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/cam_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/cam_test/deployments","created_at":"2020-01-21T08:21:49Z","updated_at":"2020-01-21T11:46:02Z","pushed_at":"2020-01-21T11:46:00Z","git_url":"git://github.com/hu-weihua/cam_test.git","ssh_url":"git@github.com:hu-weihua/cam_test.git","clone_url":"https://github.com/hu-weihua/cam_test.git","svn_url":"https://github.com/hu-weihua/cam_test","homepage":null,"size":13,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":231495393,"node_id":"MDEwOlJlcG9zaXRvcnkyMzE0OTUzOTM=","name":"Ansible_projects","full_name":"hu-weihua/Ansible_projects","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/Ansible_projects","description":"To save some ansible projects","fork":false,"url":"https://api.github.com/repos/hu-weihua/Ansible_projects","forks_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/forks","keys_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/teams","hooks_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/events","assignees_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/tags","blobs_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/subscription","commits_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/merges","archive_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/downloads","issues_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/Ansible_projects/deployments","created_at":"2020-01-03T02:20:16Z","updated_at":"2020-01-03T09:09:29Z","pushed_at":"2020-01-03T09:09:27Z","git_url":"git://github.com/hu-weihua/Ansible_projects.git","ssh_url":"git@github.com:hu-weihua/Ansible_projects.git","clone_url":"https://github.com/hu-weihua/Ansible_projects.git","svn_url":"https://github.com/hu-weihua/Ansible_projects","homepage":null,"size":19,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":88809435,"node_id":"MDEwOlJlcG9zaXRvcnk4ODgwOTQzNQ==","name":"xcat-process","full_name":"hu-weihua/xcat-process","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-process","description":"The process, guidence, rule, tips need to follow by xCAT developers and FVTs","fork":false,"url":"https://api.github.com/repos/hu-weihua/xcat-process","forks_url":"https://api.github.com/repos/hu-weihua/xcat-process/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-process/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-process/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-process/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-process/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-process/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-process/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-process/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-process/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-process/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-process/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-process/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-process/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-process/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-process/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-process/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-process/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-process/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-process/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-process/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-process/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-process/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-process/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-process/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-process/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-process/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-process/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-process/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-process/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-process/deployments","created_at":"2017-04-20T01:58:16Z","updated_at":"2019-06-25T18:55:32Z","pushed_at":"2017-04-19T09:45:48Z","git_url":"git://github.com/hu-weihua/xcat-process.git","ssh_url":"git@github.com:hu-weihua/xcat-process.git","clone_url":"https://github.com/hu-weihua/xcat-process.git","svn_url":"https://github.com/hu-weihua/xcat-process","homepage":null,"size":337,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":2,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":2,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":40430481,"node_id":"MDEwOlJlcG9zaXRvcnk0MDQzMDQ4MQ==","name":"xcat-core","full_name":"hu-weihua/xcat-core","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-core","description":"Code repo for xCAT core packages","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-core","forks_url":"https://api.github.com/repos/hu-weihua/xcat-core/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-core/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-core/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-core/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-core/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-core/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-core/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-core/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-core/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-core/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-core/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-core/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-core/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-core/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-core/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-core/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-core/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-core/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-core/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-core/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-core/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-core/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-core/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-core/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-core/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-core/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-core/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-core/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-core/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-core/deployments","created_at":"2015-08-09T08:43:37Z","updated_at":"2019-05-23T09:48:55Z","pushed_at":"2019-05-23T09:48:47Z","git_url":"git://github.com/hu-weihua/xcat-core.git","ssh_url":"git@github.com:hu-weihua/xcat-core.git","clone_url":"https://github.com/hu-weihua/xcat-core.git","svn_url":"https://github.com/hu-weihua/xcat-core","homepage":null,"size":204915,"stargazers_count":0,"watchers_count":0,"language":"Perl","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":184030584,"node_id":"MDEwOlJlcG9zaXRvcnkxODQwMzA1ODQ=","name":"xcat-docker","full_name":"hu-weihua/xcat-docker","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-docker","description":"Repository to build Docker image for xCAT","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-docker","forks_url":"https://api.github.com/repos/hu-weihua/xcat-docker/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-docker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-docker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-docker/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-docker/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-docker/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-docker/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-docker/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-docker/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-docker/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-docker/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-docker/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-docker/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-docker/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-docker/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-docker/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-docker/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-docker/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-docker/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-docker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-docker/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-docker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-docker/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-docker/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-docker/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-docker/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-docker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-docker/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-docker/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-docker/deployments","created_at":"2019-04-29T08:25:05Z","updated_at":"2019-05-17T10:33:30Z","pushed_at":"2019-05-23T10:38:53Z","git_url":"git://github.com/hu-weihua/xcat-docker.git","ssh_url":"git@github.com:hu-weihua/xcat-docker.git","clone_url":"https://github.com/hu-weihua/xcat-docker.git","svn_url":"https://github.com/hu-weihua/xcat-docker","homepage":"","size":22,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":178361168,"node_id":"MDEwOlJlcG9zaXRvcnkxNzgzNjExNjg=","name":"xcat2.github.io","full_name":"hu-weihua/xcat2.github.io","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat2.github.io","description":"The repo manages the xcat2 home page: https://xcat2.github.io/","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat2.github.io","forks_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat2.github.io/deployments","created_at":"2019-03-29T08:09:00Z","updated_at":"2019-03-29T08:09:02Z","pushed_at":"2019-03-29T08:25:09Z","git_url":"git://github.com/hu-weihua/xcat2.github.io.git","ssh_url":"git@github.com:hu-weihua/xcat2.github.io.git","clone_url":"https://github.com/hu-weihua/xcat2.github.io.git","svn_url":"https://github.com/hu-weihua/xcat2.github.io","homepage":"","size":35142,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":91755483,"node_id":"MDEwOlJlcG9zaXRvcnk5MTc1NTQ4Mw==","name":"xcat.org","full_name":"hu-weihua/xcat.org","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat.org","description":"Repository for managing the xcat.org website ","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat.org","forks_url":"https://api.github.com/repos/hu-weihua/xcat.org/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat.org/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat.org/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat.org/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat.org/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat.org/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat.org/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat.org/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat.org/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat.org/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat.org/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat.org/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat.org/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat.org/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat.org/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat.org/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat.org/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat.org/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat.org/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat.org/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat.org/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat.org/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat.org/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat.org/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat.org/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat.org/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat.org/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat.org/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat.org/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat.org/deployments","created_at":"2017-05-19T02:07:52Z","updated_at":"2018-09-27T06:11:32Z","pushed_at":"2018-09-27T06:11:31Z","git_url":"git://github.com/hu-weihua/xcat.org.git","ssh_url":"git@github.com:hu-weihua/xcat.org.git","clone_url":"https://github.com/hu-weihua/xcat.org.git","svn_url":"https://github.com/hu-weihua/xcat.org","homepage":null,"size":159,"stargazers_count":0,"watchers_count":0,"language":"HTML","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":144540798,"node_id":"MDEwOlJlcG9zaXRvcnkxNDQ1NDA3OTg=","name":"travis_ci_test","full_name":"hu-weihua/travis_ci_test","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/travis_ci_test","description":"This repository is used to test travis ci","fork":false,"url":"https://api.github.com/repos/hu-weihua/travis_ci_test","forks_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/travis_ci_test/deployments","created_at":"2018-08-13T06:54:07Z","updated_at":"2018-08-17T08:53:39Z","pushed_at":"2018-08-17T08:53:38Z","git_url":"git://github.com/hu-weihua/travis_ci_test.git","ssh_url":"git@github.com:hu-weihua/travis_ci_test.git","clone_url":"https://github.com/hu-weihua/travis_ci_test.git","svn_url":"https://github.com/hu-weihua/travis_ci_test","homepage":null,"size":22,"stargazers_count":0,"watchers_count":0,"language":"Shell","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"has_discussions":false,"forks_count":1,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":1,"open_issues":1,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":131094324,"node_id":"MDEwOlJlcG9zaXRvcnkxMzEwOTQzMjQ=","name":"goconserver","full_name":"hu-weihua/goconserver","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/goconserver","description":"Microservice based console server to help log and redirect the terminal content for multiple session hosts.","fork":true,"url":"https://api.github.com/repos/hu-weihua/goconserver","forks_url":"https://api.github.com/repos/hu-weihua/goconserver/forks","keys_url":"https://api.github.com/repos/hu-weihua/goconserver/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/goconserver/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/goconserver/teams","hooks_url":"https://api.github.com/repos/hu-weihua/goconserver/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/goconserver/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/goconserver/events","assignees_url":"https://api.github.com/repos/hu-weihua/goconserver/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/goconserver/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/goconserver/tags","blobs_url":"https://api.github.com/repos/hu-weihua/goconserver/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/goconserver/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/goconserver/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/goconserver/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/goconserver/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/goconserver/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/goconserver/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/goconserver/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/goconserver/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/goconserver/subscription","commits_url":"https://api.github.com/repos/hu-weihua/goconserver/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/goconserver/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/goconserver/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/goconserver/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/goconserver/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/goconserver/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/goconserver/merges","archive_url":"https://api.github.com/repos/hu-weihua/goconserver/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/goconserver/downloads","issues_url":"https://api.github.com/repos/hu-weihua/goconserver/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/goconserver/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/goconserver/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/goconserver/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/goconserver/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/goconserver/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/goconserver/deployments","created_at":"2018-04-26T03:21:26Z","updated_at":"2018-04-26T03:21:28Z","pushed_at":"2018-04-17T04:11:25Z","git_url":"git://github.com/hu-weihua/goconserver.git","ssh_url":"git@github.com:hu-weihua/goconserver.git","clone_url":"https://github.com/hu-weihua/goconserver.git","svn_url":"https://github.com/hu-weihua/goconserver","homepage":"","size":6913,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":131093726,"node_id":"MDEwOlJlcG9zaXRvcnkxMzEwOTM3MjY=","name":"xcat-inventory","full_name":"hu-weihua/xcat-inventory","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-inventory","description":"an inventory tool for xcat cluster","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-inventory","forks_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-inventory/deployments","created_at":"2018-04-26T03:15:17Z","updated_at":"2018-04-26T03:15:19Z","pushed_at":"2018-04-24T20:30:44Z","git_url":"git://github.com/hu-weihua/xcat-inventory.git","ssh_url":"git@github.com:hu-weihua/xcat-inventory.git","clone_url":"https://github.com/hu-weihua/xcat-inventory.git","svn_url":"https://github.com/hu-weihua/xcat-inventory","homepage":null,"size":357,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}},{"id":40395956,"node_id":"MDEwOlJlcG9zaXRvcnk0MDM5NTk1Ng==","name":"xcat-doc","full_name":"hu-weihua/xcat-doc","private":false,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/xcat-doc","description":"xCAT Documentation","fork":true,"url":"https://api.github.com/repos/hu-weihua/xcat-doc","forks_url":"https://api.github.com/repos/hu-weihua/xcat-doc/forks","keys_url":"https://api.github.com/repos/hu-weihua/xcat-doc/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/xcat-doc/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/xcat-doc/teams","hooks_url":"https://api.github.com/repos/hu-weihua/xcat-doc/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/xcat-doc/events","assignees_url":"https://api.github.com/repos/hu-weihua/xcat-doc/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/xcat-doc/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/xcat-doc/tags","blobs_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/xcat-doc/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/xcat-doc/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/xcat-doc/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/xcat-doc/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/xcat-doc/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/xcat-doc/subscription","commits_url":"https://api.github.com/repos/hu-weihua/xcat-doc/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/xcat-doc/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/xcat-doc/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/xcat-doc/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/xcat-doc/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/xcat-doc/merges","archive_url":"https://api.github.com/repos/hu-weihua/xcat-doc/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/xcat-doc/downloads","issues_url":"https://api.github.com/repos/hu-weihua/xcat-doc/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/xcat-doc/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/xcat-doc/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/xcat-doc/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/xcat-doc/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/xcat-doc/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/xcat-doc/deployments","created_at":"2015-08-08T07:59:40Z","updated_at":"2015-07-23T02:45:40Z","pushed_at":"2015-08-10T07:06:39Z","git_url":"git://github.com/hu-weihua/xcat-doc.git","ssh_url":"git@github.com:hu-weihua/xcat-doc.git","clone_url":"https://github.com/hu-weihua/xcat-doc.git","svn_url":"https://git gunicorn-web stdout | hub.com/hu-weihua/xcat-doc","homepage":null,"size":4276,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"public","forks":0,"open_issues":0,"watchers":0,"default_branch":"master","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true}}] gunicorn-web stdout | 2023-08-08 06:19:17,713 [199] [DEBUG] [app] Ending request: urn:request:1e5e0911-6efa-4520-b0bb-aaf54a42d7ec (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/sources) {'endpoint': 'api.buildtriggersources', 'request_id': 'urn:request:1e5e0911-6efa-4520-b0bb-aaf54a42d7ec', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/sources', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/sources', 'parameters': {}, 'json_body': {'namespace': 'hu-weihua'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:17,713 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:17,714 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:17 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/sources HTTP/1.0" 200 6085 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:17 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/sources HTTP/2.0" 200 6085 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.506 569 0.506) manifestbackfillworker stdout | 2023-08-08 06:19:18,371 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2023-08-08 06:19:18,372 [67] [INFO] [apscheduler.executors.default] Running job "ManifestBackfillWorker._backfill_manifests (trigger: interval[1:00:00], next run at: 2023-08-08 06:19:18 UTC)" (scheduled at 2023-08-08 06:19:18.371678+00:00) manifestbackfillworker stdout | 2023-08-08 06:19:18,373 [67] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" WHERE ("t1"."layers_compressed_size" IS %s) LIMIT %s OFFSET %s', [None, 1, 0]) manifestbackfillworker stdout | 2023-08-08 06:19:18,373 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 07:19:18.371678+00:00 (in 3599.999602 seconds) manifestbackfillworker stdout | 2023-08-08 06:19:18,379 [67] [DEBUG] [__main__] Manifest backfill worker has completed; skipping manifestbackfillworker stdout | 2023-08-08 06:19:18,379 [67] [DEBUG] [data.database] Disconnecting from database. manifestbackfillworker stdout | 2023-08-08 06:19:18,379 [67] [INFO] [apscheduler.executors.default] Job "ManifestBackfillWorker._backfill_manifests (trigger: interval[1:00:00], next run at: 2023-08-08 07:19:18 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:19:18,534 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} repositorygcworker stdout | 2023-08-08 06:19:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:19:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:04.187429+00:00 (in 44.999684 seconds) repositorygcworker stdout | 2023-08-08 06:19:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:24:19 UTC)" (scheduled at 2023-08-08 06:19:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:19:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 19, 19, 188237), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:19:19,195 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 19, 19, 188237), True, datetime.datetime(2023, 8, 8, 6, 19, 19, 188237), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:19:19,197 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 19, 19, 188237), True, datetime.datetime(2023, 8, 8, 6, 19, 19, 188237), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 19, 19, 188237), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:19:19,198 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:19:19,198 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:24:19 UTC)" executed successfully gcworker stdout | 2023-08-08 06:19:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:19:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:29.417660+00:00 (in 9.999472 seconds) gcworker stdout | 2023-08-08 06:19:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:29 UTC)" (scheduled at 2023-08-08 06:19:19.417660+00:00) gcworker stdout | 2023-08-08 06:19:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265959418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:19:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:19:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:19:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:19:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:19:19,687 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:26.890741+00:00 (in 7.202920 seconds) securityworker stdout | 2023-08-08 06:19:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:49 UTC)" (scheduled at 2023-08-08 06:19:19.687363+00:00) securityworker stdout | 2023-08-08 06:19:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:19:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:19:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:19:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:19:19,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:19:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:19:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 14, 19, 692003), 1, 2]) securityworker stdout | 2023-08-08 06:19:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 14, 19, 692003), 1, 2]) securityworker stdout | 2023-08-08 06:19:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:19:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:19:19,815 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:19:20,323 [197] [DEBUG] [app] Starting request: urn:request:2173e33b-c047-4e70-a9fc-349fac084a58 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/subdir) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:20,323 [200] [DEBUG] [app] Starting request: urn:request:0082d7ea-fa97-42d7-86b8-168566f4312a (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/fields/refs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:20,323 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,323 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,323 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,323 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,324 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,324 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,330 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:20,330 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:20,330 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:20,330 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,330 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:20,330 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,330 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:20,330 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,330 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:20,330 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:20,330 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:20,330 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,330 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:20,331 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:20,331 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:20,331 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:20,333 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:20,333 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:20,339 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:20,339 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:20,340 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,340 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,343 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,343 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,345 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,345 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,347 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,348 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:20,349 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:20,349 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:20,351 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:20,474 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:20,474 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4937', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '63', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E22E:579B:F5CBA1:1F7A899:64D1DE68'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAQJAIKSIKWIRR2X76TE2HPZI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:20,478 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:20,479 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4936', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '64', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E28C:3795:E5D608:1D820A3:64D1DE68'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAQJAIKSIKWIRR2X76TE2HPZI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:20,481 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:20,540 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:20,541 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"c97c4742e5d29a93022e99f19f25e436ecd0a03e29a447348651a030e34bfcd8"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4935', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '65', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E22E:579B:F5CBBE:1F7A8D5:64D1DE68'} [{"name":"branch1","commit":{"sha":"1d34effde035584623f29666b8f41b42e020ad4a","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/1d34effde035584623f29666b8f41b42e020ad4a"},"protected":false},{"name":"branch2","commit":{"sha":"c6ef6983d9344c211af3fc7312bddccbe7c0dd7c","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/c6ef6983d9344c211af3fc7312bddccbe7c0dd7c"},"protected":false},{"name":"hu-weihua-patch-1","commit":{"sha":"7f19019c0b0bf7eb48b55479771b213e8081c884","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/7f19019c0b0bf7eb48b55479771b213e8081c884"},"protected":false},{"name":"hu-weihua-patch-2","commit":{"sha":"65e01fcd85dfe53a2d52b5f7e188fca3f30daf78","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/65e01fcd85dfe53a2d52b5f7e188fca3f30daf78"},"protected":false},{"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d"},"protected":false}] gunicorn-web stdout | 2023-08-08 06:19:20,546 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:20,625 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:20,626 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4934', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '66', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E292:494F:DCF201:1C65B5E:64D1DE68'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAQJAIKSIKWIRR2X76TE2HPZI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:20,671 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:20,672 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4933', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '67', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E294:5801:EC730F:1E4EA85:64D1DE68'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAQJAIKSIKWIRR2X76TE2HPZI","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:20,711 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:20,712 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"c97c4742e5d29a93022e99f19f25e436ecd0a03e29a447348651a030e34bfcd8"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4932', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '68', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E292:494F:DCF223:1C65BAE:64D1DE68'} [{"name":"branch1","commit":{"sha":"1d34effde035584623f29666b8f41b42e020ad4a","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/1d34effde035584623f29666b8f41b42e020ad4a"},"protected":false},{"name":"branch2","commit":{"sha":"c6ef6983d9344c211af3fc7312bddccbe7c0dd7c","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/c6ef6983d9344c211af3fc7312bddccbe7c0dd7c"},"protected":false},{"name":"hu-weihua-patch-1","commit":{"sha":"7f19019c0b0bf7eb48b55479771b213e8081c884","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/7f19019c0b0bf7eb48b55479771b213e8081c884"},"protected":false},{"name":"hu-weihua-patch-2","commit":{"sha":"65e01fcd85dfe53a2d52b5f7e188fca3f30daf78","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/65e01fcd85dfe53a2d52b5f7e188fca3f30daf78"},"protected":false},{"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d"},"protected":false}] gunicorn-web stdout | 2023-08-08 06:19:20,750 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/tags HTTP/1.1" 200 2 gunicorn-web stdout | 2023-08-08 06:19:20,750 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/tags {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '2', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"1c5b2577e68f49fa40e894297cb1a424cc34cc7d391eb266ad138e5dc0751561"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4931', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '69', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': 'E294:5801:EC7329:1E4EAB4:64D1DE68'} [] gunicorn-web stdout | 2023-08-08 06:19:20,752 [200] [DEBUG] [app] Ending request: urn:request:0082d7ea-fa97-42d7-86b8-168566f4312a (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/fields/refs) {'endpoint': 'api.buildtriggerfieldvalues', 'request_id': 'urn:request:0082d7ea-fa97-42d7-86b8-168566f4312a', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/fields/refs', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/fields/refs', 'parameters': {}, 'json_body': {'build_source': 'hu-weihua/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:20,753 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:20,753 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:20 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/fields/refs HTTP/1.0" 200 225 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:20 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/fields/refs HTTP/2.0" 200 225 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.432 501 0.433) gunicorn-web stdout | 2023-08-08 06:19:20,802 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/branches/main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:20,803 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/branches/main {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"f6887f58b54edc91213c1d73d63e98c6fb77cb77eec99571a93a0a96e8434708"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4930', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '70', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E28C:3795:E5D67D:1D8218F:64D1DE68'} {"name":"main","commit":{"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","node_id":"C_kwDOG6AIe9oAKDg4MGZiNzAwZjhiNTEwM2U3YWRjMjBmMDgwMzg0MzRlOWY3MTJjNWQ","commit":{"author":{"name":"Hu Weihua","email":"hu_weihua@163.com","date":"2023-08-08T02:50:20Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2023-08-08T02:50:20Z"},"message":"Update update_commit.sh","tree":{"sha":"c5abbb94e6edfb389f732ea578c71d6db692406d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/c5abbb94e6edfb389f732ea578c71d6db692406d"},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsBcBAABCAAQBQJk0a1sCRBK7hj4Ov3rIwAA5MMIABciqM6Y7qngb0Joeh7FSIAp\nnv0pckL5M+0g6IBcVfxG4RC6Hfti5Gs1lYKBppBJMF5OUVdITpe2GOii6WcGieL9\nbo/Acx0X4liVp4QzoQQTNYRmCZFODrlb/FVuVcdHRtBO/hXNAsqmPOHGC1Guf5lG\nHWpLPZF9e8lnbI1/MIiu4O5+OK+qRw26pEbRHjukBa921OQ/27s+MATflSKJ+bBJ\nBshN6stBgyAA4kDsPwl7iIHn1xSGhADLw3q8BnDRIT5jRHzDoqHdck9VpVatNxxL\nK+9HZiaHhyQ3+CsvT5/NNMy5cWoqJvSpjNobgmw4bYg70AB2xUHy85jPOl7Rizk=\n=DSm6\n-----END PGP SIGNATURE-----\n","payload":"tree c5abbb94e6edfb389f732ea578c71d6db692406d\nparent f0030ac4a54f1c210daf84729cfd233407d43d14\nauthor Hu Weihua 1691463020 +0800\ncommitter GitHub 1691463020 +0800\n\nUpdate update_commit.sh"}},"url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d","html_url":"https://github.com/hu-weihua/quay_build_test/commit/880fb700f8b5103e7adc20f08038434e9f712c5d","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/880fb700f8b5103e7adc20f08038434e9f712c5d/comments","author":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","site_admin":false},"parents":[{"sha":"f0030ac4a54f1c210daf84729cfd233407d43d14","url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits/f0030ac4a54f1c210daf84729cfd233407d43d14","html_url":"https://github.com/hu-weihua/quay_build_test/commit/f0030ac4a54f1c210daf84729cfd233407d43d14"}]},"_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/branches/main","html":"https://github.com/hu-weihua/quay_build_test/tree/main"},"protected":false,"protection":{"enabled":false,"required_status_checks":{"enforcement_level":"off","contexts":[],"checks":[]}},"protection_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches/main/protection"} manifestbackfillworker stdout | 2023-08-08 06:19:20,848 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:19:20,950 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d?recursive=1 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:20,951 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d?recursive=1 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:20 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=86400, s-maxage=86400', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"aaff743540482f0d6e451515b52125a566733b6b42b0492704f09d9b2904b71a"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4929', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '71', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'E28C:3795:E5D69D:1D821CE:64D1DE68'} {"sha":"880fb700f8b5103e7adc20f08038434e9f712c5d","url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees/880fb700f8b5103e7adc20f08038434e9f712c5d","tree":[{"path":"Dockerfile","mode":"100644","type":"blob","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6"},{"path":"README.md","mode":"100644","type":"blob","sha":"929a3f9add2efe6f5d73c5226dc603686c63a202","size":57,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/929a3f9add2efe6f5d73c5226dc603686c63a202"},{"path":"aaa","mode":"100644","type":"blob","sha":"73cf2b7c923ab8e99ff19403d3b7c4191b2460a3","size":20,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/73cf2b7c923ab8e99ff19403d3b7c4191b2460a3"},{"path":"newfile5540167606144473","mode":"100644","type":"blob","sha":"aa5d92554efab5c3075f098e51bd8dc9642d6075","size":22,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/aa5d92554efab5c3075f098e51bd8dc9642d6075"},{"path":"newfile7189991658911539","mode":"100644","type":"blob","sha":"894bdc5476f8f12f6e90c706cc91aa84cf327231","size":22,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/894bdc5476f8f12f6e90c706cc91aa84cf327231"},{"path":"update_commit.sh","mode":"100644","type":"blob","sha":"8ac65cd6da3c09ee9b225fddc5eea5023f65c6aa","size":69,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/8ac65cd6da3c09ee9b225fddc5eea5023f65c6aa"}],"truncated":false} gunicorn-web stdout | 2023-08-08 06:19:20,953 [197] [DEBUG] [app] Ending request: urn:request:2173e33b-c047-4e70-a9fc-349fac084a58 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/subdir) {'endpoint': 'api.buildtriggersubdirs', 'request_id': 'urn:request:2173e33b-c047-4e70-a9fc-349fac084a58', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/subdir', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/subdir', 'parameters': {}, 'json_body': {'build_source': 'hu-weihua/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:20,954 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:20,955 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:20 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/subdir HTTP/1.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:20 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/subdir HTTP/2.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.633 150 0.634) storagereplication stdout | 2023-08-08 06:19:21,305 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:19:22,179 [200] [DEBUG] [app] Starting request: urn:request:96cca211-32ad-4818-b064-4e337fb7092c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:19:22,180 [197] [DEBUG] [app] Starting request: urn:request:a6577ccf-d848-4c5d-a8d7-c2dfbcfc9547 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:19:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:22,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:22,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:19:22,185 [203] [DEBUG] [app] Starting request: urn:request:8ed4b19c-27e0-4d9f-b530-c6ebc8c82ded (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:19:22,186 [203] [DEBUG] [app] Ending request: urn:request:8ed4b19c-27e0-4d9f-b530-c6ebc8c82ded (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8ed4b19c-27e0-4d9f-b530-c6ebc8c82ded', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:19:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:19:22,186 [209] [DEBUG] [app] Starting request: urn:request:7c3d15aa-2f09-4884-9501-d4c65d00496c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:19:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:19:22,187 [209] [DEBUG] [app] Ending request: urn:request:7c3d15aa-2f09-4884-9501-d4c65d00496c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7c3d15aa-2f09-4884-9501-d4c65d00496c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:19:22,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:19:22,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:22,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:22,191 [197] [DEBUG] [app] Starting request: urn:request:f0887b04-3770-4dee-b166-2896f5ed610e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:19:22,191 [197] [DEBUG] [app] Ending request: urn:request:f0887b04-3770-4dee-b166-2896f5ed610e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f0887b04-3770-4dee-b166-2896f5ed610e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:19:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:19:22,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:22,193 [199] [DEBUG] [app] Starting request: urn:request:8c38da76-95d8-46d8-a99c-3ebb10a32736 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:19:22,194 [199] [DEBUG] [app] Ending request: urn:request:8c38da76-95d8-46d8-a99c-3ebb10a32736 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8c38da76-95d8-46d8-a99c-3ebb10a32736', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:19:22,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:19:22,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:22,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:19:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:19:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:19:22,202 [200] [DEBUG] [app] Ending request: urn:request:96cca211-32ad-4818-b064-4e337fb7092c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:96cca211-32ad-4818-b064-4e337fb7092c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:19:22,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:22,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:19:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:19:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:19:22,203 [197] [DEBUG] [app] Ending request: urn:request:a6577ccf-d848-4c5d-a8d7-c2dfbcfc9547 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a6577ccf-d848-4c5d-a8d7-c2dfbcfc9547', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:19:22,204 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:19:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:19:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:19:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:19:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:19:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:28.874906+00:00 (in 6.580634 seconds) chunkcleanupworker stdout | 2023-08-08 06:19:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:32 UTC)" (scheduled at 2023-08-08 06:19:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:19:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:19:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 22, 294886), True, datetime.datetime(2023, 8, 8, 6, 19, 22, 294886), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:19:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:19:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:19:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:32 UTC)" executed successfully builder stdout | 2023-08-08 06:19:23,732 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:19:23,740 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:19:23,741 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:19:23,745 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:19:23,985 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:19:25,188 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:19:25,193 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:19:25,195 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:19:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:19:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:35.371682+00:00 (in 9.999471 seconds) notificationworker stdout | 2023-08-08 06:19:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:35 UTC)" (scheduled at 2023-08-08 06:19:25.371682+00:00) notificationworker stdout | 2023-08-08 06:19:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:19:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 25, 372911), True, datetime.datetime(2023, 8, 8, 6, 19, 25, 372911), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:19:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:19:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:19:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:19:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:19:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:47.398551+00:00 (in 21.883686 seconds) exportactionlogsworker stdout | 2023-08-08 06:19:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:25 UTC)" (scheduled at 2023-08-08 06:19:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:19:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:19:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:19:25,802 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:19:26,156 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:19:26,336 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:19:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:19:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:49.687363+00:00 (in 22.796136 seconds) securityworker stdout | 2023-08-08 06:19:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:56 UTC)" (scheduled at 2023-08-08 06:19:26.890741+00:00) securityworker stdout | 2023-08-08 06:19:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:19:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:19:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:19:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:19:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:19:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:19:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:19:26,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:19:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:19:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:19:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 14, 26, 896789), 1, 2]) securityworker stdout | 2023-08-08 06:19:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 14, 26, 896789), 1, 2]) securityworker stdout | 2023-08-08 06:19:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:19:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:19:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:19:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:19:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:19:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:19:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:19:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:19:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:27,167 [200] [DEBUG] [app] Starting request: urn:request:b71f607c-e0a3-4c23-9933-cc4be1d9d62e (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:27,167 [197] [DEBUG] [app] Starting request: urn:request:bd69ba51-57aa-465c-a74c-124c39764291 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:27,167 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,167 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,167 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,167 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,168 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,168 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,174 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:27,174 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:27,174 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,174 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:27,174 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:27,174 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,174 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:27,175 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,175 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,175 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,175 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,175 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:27,175 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,175 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,175 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,175 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,177 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:27,177 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:27,183 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,183 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,184 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,184 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,187 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,187 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,190 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,190 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,192 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,192 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,193 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:19:27,194 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:19:27,195 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,195 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,195 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,196 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,196 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,196 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,197 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:27,198 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 teamsyncworker stdout | 2023-08-08 06:19:27,287 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:19:27,335 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:27,336 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:27 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4928', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '72', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DAD6:5E74:E7F22A:1DC3589:64D1DE6F'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPASC6XS5STR46W4VPV3E2HPZW","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:27,341 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:27,342 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:27 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4927', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '73', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DAC8:4043:E48AFC:1D53FC4:64D1DE6F'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPASC6XS5STR46W4VPV3E2HPZW","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:27,421 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/contents/d HTTP/1.1" 404 None gunicorn-web stdout | 2023-08-08 06:19:27,422 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/contents/d {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 404 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:27 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4926', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '74', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'vary': 'Accept-Encoding, Accept, X-Requested-With', 'content-encoding': 'gzip', 'x-github-request-id': 'DAC8:4043:E48B27:1D5400A:64D1DE6F'} {"message":"Not Found","documentation_url":"https://docs.github.com/rest/repos/contents#get-repository-content"} gunicorn-web stdout | 2023-08-08 06:19:27,422 [200] [ERROR] [buildtrigger.githubhandler] got error from trying to find github file 404 {"message": "Not Found", "documentation_url": "https://docs.github.com/rest/repos/contents#get-repository-content"} gunicorn-web stdout | 2023-08-08 06:19:27,424 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:19:27,426 [200] [DEBUG] [app] Ending request: urn:request:b71f607c-e0a3-4c23-9933-cc4be1d9d62e (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:b71f607c-e0a3-4c23-9933-cc4be1d9d62e', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': 'd', 'context': None}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:27,427 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:27 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze HTTP/2.0" 200 214 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.262 203 0.262) gunicorn-web stdout | 2023-08-08 06:19:27,427 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:27 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze HTTP/1.0" 200 214 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:27,460 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/contents/Dockerfile HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:27,461 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:27 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"35a89d24e0791670dc4732e5402e3a749e89c5e6"', 'last-modified': 'Tue, 08 Aug 2023 02:50:20 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4925', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '75', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DAD6:5E74:E7F24C:1DC35D9:64D1DE6F'} {"name":"Dockerfile","path":"Dockerfile","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","html_url":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile","git_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","download_url":"https://raw.githubusercontent.com/hu-weihua/quay_build_test/main/Dockerfile?token=ABYHPAW2CI27H664NMYKKR3E2HPKW","type":"file","content":"RlJPTSByZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbS91Ymk3L3ViaQoKVVNF\nUiByb290CkxBQkVMIG1haW50YWluZXI9IndodSIKCiMgVXBkYXRlIGltYWdl\nClJVTiB5dW0gdXBkYXRlIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24t\nbWFuYWdlciAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KUlVOIHl1bSBp\nbnN0YWxsIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24tbWFuYWdlciBo\ndHRwZCAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KCiMgQWRkIGRlZmF1\nbHQgV2ViIHBhZ2UgYW5kIGV4cG9zZSBwb3J0ClJVTiBlY2hvICJUaGUgV2Vi\nIFNlcnZlciBpcyBSdW5uaW5nIiA+IC92YXIvd3d3L2h0bWwvaW5kZXguaHRt\nbApSVU4gZWNobyAicHVzaCB0ZXN0IiA+IC93aHUudHh0ClJVTiBlY2hvICJw\ndXNoIHRlc3QxIiA+IC93aHUudHh0CkVYUE9TRSA4MAoKIyBTdGFydCB0aGUg\nc2VydmljZQpDTUQgWyItRCIsICJGT1JFR1JPVU5EIl0KRU5UUllQT0lOVCBb\nIi91c3Ivc2Jpbi9odHRwZCJdCg==\n","encoding":"base64","_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","git":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","html":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile"}} gunicorn-web stdout | 2023-08-08 06:19:27,463 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:19:27,466 [197] [DEBUG] [app] Ending request: urn:request:bd69ba51-57aa-465c-a74c-124c39764291 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:bd69ba51-57aa-465c-a74c-124c39764291', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': '', 'context': None}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:27,466 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:27,467 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:27 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze HTTP/1.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:27 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze HTTP/2.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.302 1117 0.301) gunicorn-web stdout | 2023-08-08 06:19:27,512 [197] [DEBUG] [app] Starting request: urn:request:270b6e22-b0ec-4003-a3fc-18d797ba74c5 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:27,512 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,512 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,513 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,519 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:27,519 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:27,519 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,519 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:27,519 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,519 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,519 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,520 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,522 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:27,528 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,529 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,531 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,534 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:27,536 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:27,538 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:19:27,539 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,539 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,539 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:19:27,541 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:27,592 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:19:27,677 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:27,678 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:27 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4924', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '76', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DAE0:6710:109D3D8:21F304C:64D1DE6F'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPASC6XS5STR46W4VPV3E2HPZW","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:27,728 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:19:27,765 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/contents/Dockerfile HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:27,766 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:27 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"35a89d24e0791670dc4732e5402e3a749e89c5e6"', 'last-modified': 'Tue, 08 Aug 2023 02:50:20 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4923', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '77', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DAE0:6710:109D403:21F30A5:64D1DE6F'} {"name":"Dockerfile","path":"Dockerfile","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","html_url":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile","git_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","download_url":"https://raw.githubusercontent.com/hu-weihua/quay_build_test/main/Dockerfile?token=ABYHPAW2CI27H664NMYKKR3E2HPKW","type":"file","content":"RlJPTSByZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbS91Ymk3L3ViaQoKVVNF\nUiByb290CkxBQkVMIG1haW50YWluZXI9IndodSIKCiMgVXBkYXRlIGltYWdl\nClJVTiB5dW0gdXBkYXRlIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24t\nbWFuYWdlciAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KUlVOIHl1bSBp\nbnN0YWxsIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24tbWFuYWdlciBo\ndHRwZCAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KCiMgQWRkIGRlZmF1\nbHQgV2ViIHBhZ2UgYW5kIGV4cG9zZSBwb3J0ClJVTiBlY2hvICJUaGUgV2Vi\nIFNlcnZlciBpcyBSdW5uaW5nIiA+IC92YXIvd3d3L2h0bWwvaW5kZXguaHRt\nbApSVU4gZWNobyAicHVzaCB0ZXN0IiA+IC93aHUudHh0ClJVTiBlY2hvICJw\ndXNoIHRlc3QxIiA+IC93aHUudHh0CkVYUE9TRSA4MAoKIyBTdGFydCB0aGUg\nc2VydmljZQpDTUQgWyItRCIsICJGT1JFR1JPVU5EIl0KRU5UUllQT0lOVCBb\nIi91c3Ivc2Jpbi9odHRwZCJdCg==\n","encoding":"base64","_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","git":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","html":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile"}} gunicorn-web stdout | 2023-08-08 06:19:27,769 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:19:27,771 [197] [DEBUG] [app] Ending request: urn:request:270b6e22-b0ec-4003-a3fc-18d797ba74c5 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:270b6e22-b0ec-4003-a3fc-18d797ba74c5', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': None}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:27,772 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:27,772 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:27 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze HTTP/1.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:27 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze HTTP/2.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.262 212 0.262) gunicorn-web stdout | 2023-08-08 06:19:27,786 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:19:27,822 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:19:27,826 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:19:27,914 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:19:27,966 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:19:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:19:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:32.293757+00:00 (in 3.418318 seconds) chunkcleanupworker stdout | 2023-08-08 06:19:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:28 UTC)" (scheduled at 2023-08-08 06:19:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:19:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:19:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:28 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:19:29,363 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} gcworker stdout | 2023-08-08 06:19:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:19:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:39.417660+00:00 (in 9.999485 seconds) gcworker stdout | 2023-08-08 06:19:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:39 UTC)" (scheduled at 2023-08-08 06:19:29.417660+00:00) gcworker stdout | 2023-08-08 06:19:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265969418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:19:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:19:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:19:29,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:39 UTC)" executed successfully builder stdout | 2023-08-08 06:19:29,488 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:19:30,551 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gunicorn-web stdout | 2023-08-08 06:19:30,649 [200] [DEBUG] [app] Starting request: urn:request:304664d1-15a4-45a1-b8a3-a9c8e0c580ee (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:30,649 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:30,649 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:30,650 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:30,656 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:30,657 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:30,657 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:30,657 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:30,657 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:30,657 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:30,657 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:30,657 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:30,660 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:30,666 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:30,667 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:30,669 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:30,672 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:30,675 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:30,676 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:19:30,678 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:30,678 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:19:30,678 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:19:30,680 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:30,820 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:30,821 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:30 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4922', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '78', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DAEC:9BFA:10AE59D:221FBF5:64D1DE72'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAUA2XXWW77CW5DNO7TE2HPZ4","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:30,912 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/contents/Dockerfile HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:30,912 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:30 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"35a89d24e0791670dc4732e5402e3a749e89c5e6"', 'last-modified': 'Tue, 08 Aug 2023 02:50:20 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4921', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '79', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DAEC:9BFA:10AE5D3:221FC55:64D1DE72'} {"name":"Dockerfile","path":"Dockerfile","sha":"35a89d24e0791670dc4732e5402e3a749e89c5e6","size":514,"url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","html_url":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile","git_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","download_url":"https://raw.githubusercontent.com/hu-weihua/quay_build_test/main/Dockerfile?token=ABYHPAVONISOA7P5KKAPTCLE2HPK4","type":"file","content":"RlJPTSByZWdpc3RyeS5hY2Nlc3MucmVkaGF0LmNvbS91Ymk3L3ViaQoKVVNF\nUiByb290CkxBQkVMIG1haW50YWluZXI9IndodSIKCiMgVXBkYXRlIGltYWdl\nClJVTiB5dW0gdXBkYXRlIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24t\nbWFuYWdlciAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KUlVOIHl1bSBp\nbnN0YWxsIC0tZGlzYWJsZXBsdWdpbj1zdWJzY3JpcHRpb24tbWFuYWdlciBo\ndHRwZCAteSAmJiBybSAtcmYgL3Zhci9jYWNoZS95dW0KCiMgQWRkIGRlZmF1\nbHQgV2ViIHBhZ2UgYW5kIGV4cG9zZSBwb3J0ClJVTiBlY2hvICJUaGUgV2Vi\nIFNlcnZlciBpcyBSdW5uaW5nIiA+IC92YXIvd3d3L2h0bWwvaW5kZXguaHRt\nbApSVU4gZWNobyAicHVzaCB0ZXN0IiA+IC93aHUudHh0ClJVTiBlY2hvICJw\ndXNoIHRlc3QxIiA+IC93aHUudHh0CkVYUE9TRSA4MAoKIyBTdGFydCB0aGUg\nc2VydmljZQpDTUQgWyItRCIsICJGT1JFR1JPVU5EIl0KRU5UUllQT0lOVCBb\nIi91c3Ivc2Jpbi9odHRwZCJdCg==\n","encoding":"base64","_links":{"self":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/Dockerfile?ref=main","git":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs/35a89d24e0791670dc4732e5402e3a749e89c5e6","html":"https://github.com/hu-weihua/quay_build_test/blob/main/Dockerfile"}} gunicorn-web stdout | 2023-08-08 06:19:30,915 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:19:30,918 [200] [DEBUG] [app] Ending request: urn:request:304664d1-15a4-45a1-b8a3-a9c8e0c580ee (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:304664d1-15a4-45a1-b8a3-a9c8e0c580ee', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:30,918 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:30,919 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:30 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze HTTP/1.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:30 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/analyze HTTP/2.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.272 554 0.271) exportactionlogsworker stdout | 2023-08-08 06:19:31,402 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:19:31,548 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:19:31,964 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:19:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:19:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:42.293757+00:00 (in 9.999459 seconds) chunkcleanupworker stdout | 2023-08-08 06:19:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:42 UTC)" (scheduled at 2023-08-08 06:19:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:19:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:19:32,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 32, 294843), True, datetime.datetime(2023, 8, 8, 6, 19, 32, 294843), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:19:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:19:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:19:32,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:42 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:19:32,460 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:19:32,785 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:19:33,098 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:19:33,486 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:19:33,491 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:19:33,494 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:19:33,497 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:19:33,500 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:19:33,511 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:19:33,519 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:19:33,522 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:19:33,526 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:19:33,755 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:19:33,764 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:19:33,764 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:19:33,768 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:34,139 [197] [DEBUG] [app] Starting request: urn:request:1c857856-8cd7-4d01-9156-84a0a6c9bb53 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/activate) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:34,139 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:34,139 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:34,140 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,146 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:34,146 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:34,146 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:34,146 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:34,146 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:34,146 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:34,146 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:34,146 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:34,149 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:34,155 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:34,156 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,159 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,162 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,165 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,166 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:34,166 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['write', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,168 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,169 [197] [DEBUG] [peewee] ('INSERT INTO "accesstoken" ("friendly_name", "token_name", "token_code", "repository_id", "created", "role_id", "temporary") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "accesstoken"."id"', ['Build Trigger: github', 'O971568QK6MOX8J0I4R6DB3XWH3LI6LI', 'v0$$+eXkBvQ442UDIingVjejz849MmoBL72rmYoOGiak36ts/R/pXLywFnBAFA/qvMz4+bsCBZaXgF6Wsp1Uzg==', 4, datetime.datetime(2023, 8, 8, 6, 19, 34, 169536), 2, False]) gunicorn-web stdout | 2023-08-08 06:19:34,175 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:34,315 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:34,316 [197] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:34 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4920', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '80', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': '8C7E:729B:F25038:1EFC790:64D1DE76'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAUYNOT2H7ZNGEX7Y5TE2HP2E","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:34,565 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "POST /repos/hu-weihua/quay_build_test/keys HTTP/1.1" 201 622 gunicorn-web stdout | 2023-08-08 06:19:34,566 [197] [DEBUG] [github.Requester] POST https://api.github.com/repos/hu-weihua/quay_build_test/keys {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python', 'Content-Type': 'application/json'} {"title": "Red Hat Quay Builder", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/MC598ROqhikoUsgPCEVjPQKyDXjDbMPPf1pSgM/YOSOI9zl48jBCJ25CJ77gRCfmZqEF7BQrlaWpy7kTvZdiT9/RrZF9U3BFF6NmhCuA86lGg9o624dnbiK0pTn1ajiofCdnr/ZJgVoY1Gcr++SapnqXzaG4/pUmygb9vk5AodBGhIL40bdN/4jZYfd5tEnfgKeTUC7umCymG69zZWVLNQfQ8H97Ggxr2jHrJDRRN7pLNYz2dLHXMe8FcMZ+uW5UT9y/CnlhlGBNRDbzOcd3pSrodk3FBLK9UXITxAzma40iEP8+ci74RJ4AQgjvnP2TfBdmtq5OXEgY0j7lLvaJ", "read_only": false} ==> 201 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:34 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '622', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"b1465e50d9b7e6404e609fd5a12418002519bc58889f535f8b21ee258265a97f"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'location': 'https://api.github.com/repos/hu-weihua/quay_build_test/keys/85931051', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4919', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '81', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': '8C7E:729B:F2506D:1EFC803:64D1DE76'} {"id":85931051,"key":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/MC598ROqhikoUsgPCEVjPQKyDXjDbMPPf1pSgM/YOSOI9zl48jBCJ25CJ77gRCfmZqEF7BQrlaWpy7kTvZdiT9/RrZF9U3BFF6NmhCuA86lGg9o624dnbiK0pTn1ajiofCdnr/ZJgVoY1Gcr++SapnqXzaG4/pUmygb9vk5AodBGhIL40bdN/4jZYfd5tEnfgKeTUC7umCymG69zZWVLNQfQ8H97Ggxr2jHrJDRRN7pLNYz2dLHXMe8FcMZ+uW5UT9y/CnlhlGBNRDbzOcd3pSrodk3FBLK9UXITxAzma40iEP8+ci74RJ4AQgjvnP2TfBdmtq5OXEgY0j7lLvaJ","url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys/85931051","title":"Red Hat Quay Builder","verified":true,"created_at":"2023-08-08T06:19:34Z","read_only":false,"last_used":null,"added_by":"hu-weihua"} gunicorn-web stdout | 2023-08-08 06:19:34,741 [197] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "POST /repos/hu-weihua/quay_build_test/hooks HTTP/1.1" 201 853 gunicorn-web stdout | 2023-08-08 06:19:34,741 [197] [DEBUG] [github.Requester] POST https://api.github.com/repos/hu-weihua/quay_build_test/hooks {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python', 'Content-Type': 'application/json'} {"name": "web", "config": {"url": "https://$token:O971568QK6MOX8J0I4R6DB3XWH3LI6LILN1YSO8YKV90SE8S49TJLPWH1CEO2QS6@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5", "content_type": "json"}} ==> 201 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:34 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '853', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': '"dafd0e70167df707c8cf449b4c8f7bc2e5b2ef0944eeab6a62741aa6a3f4c333"', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:repo_hook, repo, write:repo_hook', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'location': 'https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4918', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '82', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'x-github-request-id': '8C7E:729B:F250BB:1EFC893:64D1DE76'} {"type":"Repository","id":427820277,"name":"web","active":true,"events":["push"],"config":{"content_type":"json","insecure_ssl":"0","url":"https://$token:O971568QK6MOX8J0I4R6DB3XWH3LI6LILN1YSO8YKV90SE8S49TJLPWH1CEO2QS6@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5"},"updated_at":"2023-08-08T06:19:34Z","created_at":"2023-08-08T06:19:34Z","url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277","test_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277/test","ping_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277/pings","deliveries_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277/deliveries","last_response":{"code":null,"status":"unused","message":null}} gunicorn-web stdout | 2023-08-08 06:19:34,744 [197] [DEBUG] [peewee] ('UPDATE "repositorybuildtrigger" SET "uuid" = %s, "service_id" = %s, "repository_id" = %s, "connected_user_id" = %s, "secure_auth_token" = %s, "secure_private_key" = %s, "fully_migrated" = %s, "config" = %s, "write_token_id" = %s, "pull_robot_id" = %s, "enabled" = %s, "disabled_reason_id" = %s, "disabled_datetime" = %s, "successive_failure_count" = %s, "successive_internal_error_count" = %s WHERE ("repositorybuildtrigger"."id" = %s)', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 4, 3, 'v0$$O3aMXM8I1fqtTLpzWPYxULYnjaRrZDsbCG0gxW324/LHmRpcy4qsmB9/TIGS2uV//pjwWcIFKFpghI/qyrWNzz263aBy', 'v0$$9GB37CrNExDWypDLul4crJ+l2hwwbiybbguokk7IVJBoQHBcaFxoKZU8ker2+pvStA6aJUIxvrZSsseyIBzIe+3FlSCIav4e4lXoubp4tiPlF5aY9Qm94NKNR9VTIKCKmI7C3UyuQGMpihUjr/oeX3WImSC/ShfTYDupmu2YBp+3fzGapdhfwwEaBj0hVnV0zNXg1poKB9BhWfWERM+govZ2Qu6mrxsa21Lw1An+dpUYQEA+k/1ncAqUGGxI61w0+yecU58l8TCGdu6RQTKvnbOAuZfX9er0/pl2MJ5FNIhn3yv2G96stMcdYhxLBqVIf+FTWotzCJVYvwJCKK7Vlsqot6gOiXpq3pQ9mL90/ERQzZgDKonSu1oRCYX4WbXE46SE7qygdqmXR7gf+Oatq6PTffLPBitlV1QMnIbFbZyvvIlD5POk3/kqlpSqiwUF49udox0pZG6b2yEfl7rdNuoNlZxE/bLkJV6lI2lBxJtHjXyQ/qlD/BooudHKLHfO7yVWAVBK7qn7Epcol+QukGyLQ+TtnQ62CAndZJlgepd/sEAhYbxbwVQyxbWOm70VSB8WU7H3t1b4zhA2NJmcCskY8KziUhzdrB/ClE4+97xruclOqtvdLXvEGpFJmu7XzJNsl1ZLn8CrRYZfDIswpake9145TAmjXe4FTp8N6Ihr5OD1JWqDGCHxGLbm6ZG9RjlBnUf+LrojCu934vqp5VNu49JkQ6fx9kI5Sio3f5jhqxGY+aQoqHoB4U+WgVVIMHhTqPUVXoHl54r9QnDL1PIe5DRqtqN/5k5O9ufGyr5d5RT46U2JKM1FmdE3mmjORH9YQ7Qc3LHiYFnR45sDQd4lw7Mbt3aUGyOKDjl3fi80v28KIaXC2W7mkqW2mJPK7zEcUJ9bXRpyEsAsBPQKubWGgLyA6mPEaBph/3PtUiyApbX6HtCLv8sgMRuk7axcSTv8p5CgcIRler0QmfHe9hV/Cpah5TyMrvoLLhMwkcBRzVUL9Jwdq5CwYrd5KRqf7xhAYIadnbvlUN5Yo2N2sU80sL6p0rElB8g7FyrSe/qxXAxbmmoVezGYZ6Mhh0jyv2+GdcZojP4wbvLPD+GywEiiUpquWqGL54/dOSznoOef/WyNx6RiuuAd6sCIjurMwRfmyzEf+h4o7UzBNT4LsQtq8MleyBinMzvNJkUl88N0mkZJGPKCKNp0F3Kxsvg6z4yc53uGPA7Kuws3TVqBsVzgsv015+AXMnqbyeOqPI70jdfoDJTYvcXiAOQlOV2fmNzv0n0UZW/3VQeXj0I6OyGDTmh77wsYac3s4Ob1N0976eo8XHEwhuM+w+J+Cg6wqFuurNiONokNKt2Cx0CquLt6c4qr4s0x7JTzFwMYFWdC+zPeotxx2OtmYqf2oyNKr5mWPU4OuI5KLBj3Kc60an2TCyODX/CyLqKhzkWhWQ365rSgdFfugAD1EoP1Zj4SdTFShcx113iok3j5yzJDLHVO4plqAQ74NnnloKiPlfgdz2Qgw3BnLp/SOtvGxfBuzoCcUs/rFbn6XczRoAPQA3QtthkB2R6StGbfwqTUqBk2WaauDa7lZ0skNUM9jvq7RKRtkTsWnPdElNSra0fyWYFykpQybOpHY3P0P7y4lb+jcy8DCyhJVeuaTdMzxXsmr+zh6DenIhUzr9GTKP6f8tNq/9SzoTEpd8vdpD0H7wrTBWN6pBqwwfwNiY1zWTAxjV9/Hbk/9wVt6b553ZOet5UVpI29NU6RvUviXjPTJbN6Epvu87Lkc/NDMNfVrLGqW3pWZh+Ywxfl5/s6UN7FDWKaZwUQINXiZusReArtH4gte6VH9L3ijHGJiOwJjzb72yMo8ecPhZNCluCmIsFBlN/Wh+3Q+IdSaW1n5gwKbifyAmtfPXGUwRjITDuWcD7u6Out1mZ5ktz5s/uFls4Y3ih0GsH0V6Kw1czkqb3/QqSzeHXV13mwrOCpq1UESOUtwj3Bo1/bVz4IlYwmM4VBtTPSSCpKZbkABGi5Ld9D3pHCkf45ntWpdMKohi5t6AhhjXgTCCHj4hUXCyHHtmhdmaudsL+jMWBsnTIxHNZOSrGDzMmLTiOrZxaod8evK0l0JF+1/fCOqTEAh3F0P1Y02kws+q1O94tSBeLwR/2C+qhrHxey+TrNuGabEJZaVsFCNV2Fj2Cjob4/UQaEOdyR4n38Y3HKq2Utufygr/hJ1T6SMZwMpTerhFRIoseIp8BuYFXqCYtPrIaH9CySC4NJilZlvIVnNDk82gCQZA==', False, '{"build_source": "hu-weihua/quay_build_test", "dockerfile_path": "/Dockerfile", "context": "/", "default_tag_from_ref": true, "latest_for_default_branch": true, "tag_templates": [], "credentials": [{"name": "SSH Public Key", "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/MC598ROqhikoUsgPCEVjPQKyDXjDbMPPf1pSgM/YOSOI9zl48jBCJ25CJ77gRCfmZqEF7BQrlaWpy7kTvZdiT9/RrZF9U3BFF6NmhCuA86lGg9o624dnbiK0pTn1ajiofCdnr/ZJgVoY1Gcr++SapnqXzaG4/pUmygb9vk5AodBGhIL40bdN/4jZYfd5tEnfgKeTUC7umCymG69zZWVLNQfQ8H97Ggxr2jHrJDRRN7pLNYz2dLHXMe8FcMZ+uW5UT9y/CnlhlGBNRDbzOcd3pSrodk3FBLK9UXITxAzma40iEP8+ci74RJ4AQgjvnP2TfBdmtq5OXEgY0j7lLvaJ"}], "deploy_key_id": 85931051, "hook_id": 427820277, "master_branch": "main"}', 6, None, True, None, datetime.datetime(2023, 8, 8, 6, 19, 13, 306528), 0, 0, 5]) gunicorn-web stdout | 2023-08-08 06:19:34,749 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,751 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:34,752 [197] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [53, 4, 3, 4, datetime.datetime(2023, 8, 8, 6, 19, 34, 750699), '10.130.2.1', '{"repo": "github", "namespace": "user1_build_test", "trigger_id": "38fa6ab9-97e0-475c-bbf6-55d5541dc9b5", "service": "github", "pull_robot": null, "config": {"build_source": "hu-weihua/quay_build_test", "dockerfile_path": "/Dockerfile", "context": "/", "default_tag_from_ref": true, "latest_for_default_branch": true, "tag_templates": [], "credentials": [{"name": "SSH Public Key", "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/MC598ROqhikoUsgPCEVjPQKyDXjDbMPPf1pSgM/YOSOI9zl48jBCJ25CJ77gRCfmZqEF7BQrlaWpy7kTvZdiT9/RrZF9U3BFF6NmhCuA86lGg9o624dnbiK0pTn1ajiofCdnr/ZJgVoY1Gcr++SapnqXzaG4/pUmygb9vk5AodBGhIL40bdN/4jZYfd5tEnfgKeTUC7umCymG69zZWVLNQfQ8H97Ggxr2jHrJDRRN7pLNYz2dLHXMe8FcMZ+uW5UT9y/CnlhlGBNRDbzOcd3pSrodk3FBLK9UXITxAzma40iEP8+ci74RJ4AQgjvnP2TfBdmtq5OXEgY0j7lLvaJ"}], "deploy_key_id": 85931051, "hook_id": 427820277, "master_branch": "main"}}']) gunicorn-web stdout | 2023-08-08 06:19:34,755 [197] [DEBUG] [app] Ending request: urn:request:1c857856-8cd7-4d01-9156-84a0a6c9bb53 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/activate) {'endpoint': 'api.buildtriggeractivate', 'request_id': 'urn:request:1c857856-8cd7-4d01-9156-84a0a6c9bb53', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/activate', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/activate', 'parameters': {}, 'json_body': {'config': {'build_source': 'hu-weihua/quay_build_test', 'dockerfile_path': '/Dockerfile', 'context': '/', 'default_tag_from_ref': True, 'latest_for_default_branch': True, 'tag_templates': [], 'credentials': [{'name': 'SSH Public Key', 'value': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/MC598ROqhikoUsgPCEVjPQKyDXjDbMPPf1pSgM/YOSOI9zl48jBCJ25CJ77gRCfmZqEF7BQrlaWpy7kTvZdiT9/RrZF9U3BFF6NmhCuA86lGg9o624dnbiK0pTn1ajiofCdnr/ZJgVoY1Gcr++SapnqXzaG4/pUmygb9vk5AodBGhIL40bdN/4jZYfd5tEnfgKeTUC7umCymG69zZWVLNQfQ8H97Ggxr2jHrJDRRN7pLNYz2dLHXMe8FcMZ+uW5UT9y/CnlhlGBNRDbzOcd3pSrodk3FBLK9UXITxAzma40iEP8+ci74RJ4AQgjvnP2TfBdmtq5OXEgY0j7lLvaJ'}], 'deploy_key_id': 85931051, 'hook_id': 427820277, 'master_branch': 'main'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:34,756 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:34 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/activate HTTP/2.0" 200 965 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.620 633 0.620) gunicorn-web stdout | 2023-08-08 06:19:34,756 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:34 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5/activate HTTP/1.0" 200 965 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:19:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:19:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:45.371682+00:00 (in 9.999463 seconds) notificationworker stdout | 2023-08-08 06:19:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:45 UTC)" (scheduled at 2023-08-08 06:19:35.371682+00:00) notificationworker stdout | 2023-08-08 06:19:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:19:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 35, 372707), True, datetime.datetime(2023, 8, 8, 6, 19, 35, 372707), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:19:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:19:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:19:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:19:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:19:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:39.579576+00:00 (in 2.999186 seconds) namespacegcworker stdout | 2023-08-08 06:19:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:36 UTC)" (scheduled at 2023-08-08 06:19:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:19:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:19:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:37,156 [197] [DEBUG] [app] Starting request: urn:request:15caed50-319f-4216-8619-dc14b8baaa0b (/api/v1/repository/user1_build_test/github/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:37,156 [200] [DEBUG] [app] Starting request: urn:request:99d6057a-e091-4106-b042-cc44fb0a19c2 (/api/v1/repository/user1_build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:37,156 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,156 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,156 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,156 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,157 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,157 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,163 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:37,164 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:37,164 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,164 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:37,164 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,164 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,164 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:37,164 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,164 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,164 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:37,164 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,164 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:37,165 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,165 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,165 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,165 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,167 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:37,167 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:37,173 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,173 [200] [DEBUG] [endpoints.api.repository] Get repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:37,173 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,173 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,174 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,175 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [4, 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,176 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [4, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:19:37,177 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,177 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,177 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,178 [200] [DEBUG] [app] Ending request: urn:request:99d6057a-e091-4106-b042-cc44fb0a19c2 (/api/v1/repository/user1_build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:99d6057a-e091-4106-b042-cc44fb0a19c2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false', 'path': '/api/v1/repository/user1_build_test/github', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:37,179 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:37,179 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [4, None, 1691475577179, False, 101, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,179 [198] [DEBUG] [app] Starting request: urn:request:d61f9e99-8c8f-4be9-997f-fe6836262049 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:19:37,179 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:37 +0000] "GET /api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false HTTP/1.0" 200 3632 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false HTTP/2.0" 200 3632 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 92 0.025) gunicorn-web stdout | 2023-08-08 06:19:37,180 [199] [DEBUG] [app] Starting request: urn:request:5000247e-6264-4b96-b8a2-13a297374582 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:19:37,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:37,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:37,182 [197] [DEBUG] [app] Ending request: urn:request:15caed50-319f-4216-8619-dc14b8baaa0b (/api/v1/repository/user1_build_test/github/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:15caed50-319f-4216-8619-dc14b8baaa0b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/user1_build_test/github/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:37,182 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:37,183 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:37 +0000] "GET /api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:37,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 535 0.029) gunicorn-web stdout | 2023-08-08 06:19:37,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:19:37,185 [206] [DEBUG] [app] Starting request: urn:request:50d8d525-b636-4c3d-a97d-bf1b60c1d93c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:19:37,186 [206] [DEBUG] [app] Ending request: urn:request:50d8d525-b636-4c3d-a97d-bf1b60c1d93c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:50d8d525-b636-4c3d-a97d-bf1b60c1d93c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:19:37,186 [209] [DEBUG] [app] Starting request: urn:request:fe118521-6154-4340-8016-133902db9c49 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:19:37,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:19:37,187 [209] [DEBUG] [app] Ending request: urn:request:fe118521-6154-4340-8016-133902db9c49 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fe118521-6154-4340-8016-133902db9c49', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:19:37,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:19:37,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:37,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:37,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:37,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:37,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:37,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:37,192 [198] [DEBUG] [app] Starting request: urn:request:14c41b11-3fc1-4744-9e02-8421f40a2c37 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:19:37,192 [198] [DEBUG] [app] Ending request: urn:request:14c41b11-3fc1-4744-9e02-8421f40a2c37 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:14c41b11-3fc1-4744-9e02-8421f40a2c37', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:19:37,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:37,193 [197] [DEBUG] [app] Starting request: urn:request:3ef2bb18-d2c0-4e04-bf24-1e6ec243d7b6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:19:37,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:37,193 [197] [DEBUG] [app] Ending request: urn:request:3ef2bb18-d2c0-4e04-bf24-1e6ec243d7b6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3ef2bb18-d2c0-4e04-bf24-1e6ec243d7b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:19:37,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:19:37,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:37,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:37,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:19:37,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:19:37,198 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:19:37,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:19:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:07.197412+00:00 (in 29.999181 seconds) buildlogsarchiver stdout | 2023-08-08 06:19:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:07 UTC)" (scheduled at 2023-08-08 06:19:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:19:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 19, 37, 198696), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,202 [198] [DEBUG] [app] Ending request: urn:request:d61f9e99-8c8f-4be9-997f-fe6836262049 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d61f9e99-8c8f-4be9-997f-fe6836262049', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:19:37,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:37,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:19:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:19:37,203 [199] [DEBUG] [app] Ending request: urn:request:5000247e-6264-4b96-b8a2-13a297374582 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5000247e-6264-4b96-b8a2-13a297374582', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:19:37,203 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:19:37,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:19:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:19:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:19:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:19:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:19:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:19:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:37,767 [198] [DEBUG] [app] Starting request: urn:request:60e258cd-c580-4c3e-9aa5-7b5a9d1889d5 (/api/v1/repository/user1_build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:37,767 [197] [DEBUG] [app] Starting request: urn:request:8a532e08-82d8-4f73-8ccc-d340db857f1c (/api/v1/repository/user1_build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:37,767 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,767 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,767 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,767 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,768 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,768 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,774 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:37,774 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:37,775 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:37,775 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:37,775 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,775 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,775 [198] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:37,775 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:37,775 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,775 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:37,775 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,775 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,775 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,775 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,775 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,775 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,777 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:37,777 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:37,784 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,784 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:37,785 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,785 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,787 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['user1_build_test', 'github']) gunicorn-web stdout | 2023-08-08 06:19:37,787 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['user1_build_test', 'github']) gunicorn-web stdout | 2023-08-08 06:19:37,790 [198] [DEBUG] [app] Ending request: urn:request:60e258cd-c580-4c3e-9aa5-7b5a9d1889d5 (/api/v1/repository/user1_build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:60e258cd-c580-4c3e-9aa5-7b5a9d1889d5', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/', 'path': '/api/v1/repository/user1_build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:37,790 [197] [DEBUG] [app] Ending request: urn:request:8a532e08-82d8-4f73-8ccc-d340db857f1c (/api/v1/repository/user1_build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:8a532e08-82d8-4f73-8ccc-d340db857f1c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/', 'path': '/api/v1/repository/user1_build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:37,791 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:37,791 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/ HTTP/2.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 322 0.025) gunicorn-web stdout | 2023-08-08 06:19:37,791 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:37 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/ HTTP/1.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:37,791 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:37 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/ HTTP/1.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/ HTTP/2.0" 200 981 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 413 0.025) gunicorn-web stdout | 2023-08-08 06:19:37,963 [200] [DEBUG] [app] Starting request: urn:request:45072728-e9e2-4a81-9410-d8eda6b93cd9 (/repository/user1_build_test/github/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:37,964 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,971 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,973 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,976 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['github', 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:37,982 [200] [DEBUG] [app] Ending request: urn:request:45072728-e9e2-4a81-9410-d8eda6b93cd9 (/repository/user1_build_test/github/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:45072728-e9e2-4a81-9410-d8eda6b93cd9', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/status?token=547673b4-0f98-4bab-8b19-e06d0dac67eb', 'path': '/repository/user1_build_test/github/status', 'parameters': {'token': '547673b4-0f98-4bab-8b19-e06d0dac67eb'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:37,983 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:37,983 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:37 +0000] "GET /repository/user1_build_test/github/status?token=547673b4-0f98-4bab-8b19-e06d0dac67eb HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:37 +0000] "GET /repository/user1_build_test/github/status?token=547673b4-0f98-4bab-8b19-e06d0dac67eb HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.022 129 0.022) gunicorn-web stdout | 2023-08-08 06:19:38,167 [200] [DEBUG] [app] Starting request: urn:request:6caae17f-e68e-4b20-935b-8d5f960d3b61 (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:38,168 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:38,168 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:38,168 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:38,175 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:38,175 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:38,175 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:38,175 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:38,175 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:38,175 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:38,175 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:38,175 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:38,178 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:38,184 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:38,185 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:38,188 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:38,192 [200] [DEBUG] [app] Ending request: urn:request:6caae17f-e68e-4b20-935b-8d5f960d3b61 (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:6caae17f-e68e-4b20-935b-8d5f960d3b61', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=3', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:38,193 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:38 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 77 0.027) gunicorn-web stdout | 2023-08-08 06:19:38,193 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:38 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:19:38,529 [200] [DEBUG] [app] Starting request: urn:request:8d73e897-182b-4927-bcbc-e139f9b9c164 (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:38,529 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:38,529 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:38,530 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:38,536 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:38,536 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:38,536 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:38,537 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:38,537 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:38,537 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:38,537 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:38,537 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:38,539 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:38,545 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:38,546 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:38,550 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 10]) gunicorn-web stdout | 2023-08-08 06:19:38,553 [200] [DEBUG] [app] Ending request: urn:request:8d73e897-182b-4927-bcbc-e139f9b9c164 (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:8d73e897-182b-4927-bcbc-e139f9b9c164', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=10', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:38,554 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:38 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 419 0.028) gunicorn-web stdout | 2023-08-08 06:19:38,555 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:38 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gcworker stdout | 2023-08-08 06:19:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:19:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:49.417660+00:00 (in 9.999466 seconds) gcworker stdout | 2023-08-08 06:19:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:49 UTC)" (scheduled at 2023-08-08 06:19:39.417660+00:00) gcworker stdout | 2023-08-08 06:19:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265979418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:19:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:19:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:19:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:19:39,579 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:19:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:54.393627+00:00 (in 14.813503 seconds) namespacegcworker stdout | 2023-08-08 06:19:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:24:39 UTC)" (scheduled at 2023-08-08 06:19:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:19:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 19, 39, 580541), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:19:39,587 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 19, 39, 580541), True, datetime.datetime(2023, 8, 8, 6, 19, 39, 580541), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:19:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 19, 39, 580541), True, datetime.datetime(2023, 8, 8, 6, 19, 39, 580541), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 19, 39, 580541), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:19:39,590 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:19:39,590 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:24:39 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:19:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:19:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:46.874674+00:00 (in 4.580397 seconds) chunkcleanupworker stdout | 2023-08-08 06:19:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:52 UTC)" (scheduled at 2023-08-08 06:19:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:19:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:19:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 42, 294778), True, datetime.datetime(2023, 8, 8, 6, 19, 42, 294778), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:19:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:19:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:19:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:52 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:19:42,916 [200] [DEBUG] [app] Starting request: urn:request:786ff321-f251-4494-a94c-f7f2c1fb2f42 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:19:42,916 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:42,916 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:42,917 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:42,923 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:19:42,923 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:19:42,923 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:42,924 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:19:42,924 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:19:42,924 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:42,924 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:42,924 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:19:42,926 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:19:42,932 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:19:42,933 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:42,936 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:42,938 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:42,943 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): api.github.com:443 gunicorn-web stdout | 2023-08-08 06:19:43,070 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:43,070 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:43 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2fe2081cc1137a7b7c2bd1ea519cc22a7eb008d7c9237591c49e143c3983caa0"', 'last-modified': 'Fri, 22 Jul 2022 08:33:18 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4917', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '83', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DA9E:0147:F334BA:1F2DB74:64D1DE7E'} {"id":463472763,"node_id":"R_kgDOG6AIew","name":"quay_build_test","full_name":"hu-weihua/quay_build_test","private":true,"owner":{"login":"hu-weihua","id":7370626,"node_id":"MDQ6VXNlcjczNzA2MjY=","avatar_url":"https://avatars.githubusercontent.com/u/7370626?v=4","gravatar_id":"","url":"https://api.github.com/users/hu-weihua","html_url":"https://github.com/hu-weihua","followers_url":"https://api.github.com/users/hu-weihua/followers","following_url":"https://api.github.com/users/hu-weihua/following{/other_user}","gists_url":"https://api.github.com/users/hu-weihua/gists{/gist_id}","starred_url":"https://api.github.com/users/hu-weihua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hu-weihua/subscriptions","organizations_url":"https://api.github.com/users/hu-weihua/orgs","repos_url":"https://api.github.com/users/hu-weihua/repos","events_url":"https://api.github.com/users/hu-weihua/events{/privacy}","received_events_url":"https://api.github.com/users/hu-weihua/received_events","type":"User","site_admin":false},"html_url":"https://github.com/hu-weihua/quay_build_test","description":"It is used to test quay build function","fork":false,"url":"https://api.github.com/repos/hu-weihua/quay_build_test","forks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/forks","keys_url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hu-weihua/quay_build_test/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hu-weihua/quay_build_test/teams","hooks_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks","issue_events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/events{/number}","events_url":"https://api.github.com/repos/hu-weihua/quay_build_test/events","assignees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/assignees{/user}","branches_url":"https://api.github.com/repos/hu-weihua/quay_build_test/branches{/branch}","tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/tags","blobs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/refs{/sha}","trees_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hu-weihua/quay_build_test/statuses/{sha}","languages_url":"https://api.github.com/repos/hu-weihua/quay_build_test/languages","stargazers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/stargazers","contributors_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contributors","subscribers_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscribers","subscription_url":"https://api.github.com/repos/hu-weihua/quay_build_test/subscription","commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/commits{/sha}","git_commits_url":"https://api.github.com/repos/hu-weihua/quay_build_test/git/commits{/sha}","comments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/comments{/number}","issue_comment_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues/comments{/number}","contents_url":"https://api.github.com/repos/hu-weihua/quay_build_test/contents/{+path}","compare_url":"https://api.github.com/repos/hu-weihua/quay_build_test/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hu-weihua/quay_build_test/merges","archive_url":"https://api.github.com/repos/hu-weihua/quay_build_test/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hu-weihua/quay_build_test/downloads","issues_url":"https://api.github.com/repos/hu-weihua/quay_build_test/issues{/number}","pulls_url":"https://api.github.com/repos/hu-weihua/quay_build_test/pulls{/number}","milestones_url":"https://api.github.com/repos/hu-weihua/quay_build_test/milestones{/number}","notifications_url":"https://api.github.com/repos/hu-weihua/quay_build_test/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hu-weihua/quay_build_test/labels{/name}","releases_url":"https://api.github.com/repos/hu-weihua/quay_build_test/releases{/id}","deployments_url":"https://api.github.com/repos/hu-weihua/quay_build_test/deployments","created_at":"2022-02-25T09:28:00Z","updated_at":"2022-07-22T08:33:18Z","pushed_at":"2023-08-08T02:50:20Z","git_url":"git://github.com/hu-weihua/quay_build_test.git","ssh_url":"git@github.com:hu-weihua/quay_build_test.git","clone_url":"https://github.com/hu-weihua/quay_build_test.git","svn_url":"https://github.com/hu-weihua/quay_build_test","homepage":null,"size":14,"stargazers_count":0,"watchers_count":0,"language":"Dockerfile","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"has_discussions":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":1,"license":null,"allow_forking":true,"is_template":false,"web_commit_signoff_required":false,"topics":[],"visibility":"private","forks":0,"open_issues":1,"watchers":0,"default_branch":"main","permissions":{"admin":true,"maintain":true,"push":true,"triage":true,"pull":true},"temp_clone_token":"ABYHPAUWCOKDN2EOSI4HOULE2HP2W","allow_squash_merge":true,"allow_merge_commit":true,"allow_rebase_merge":true,"allow_auto_merge":false,"delete_branch_on_merge":false,"allow_update_branch":false,"use_squash_pr_title_as_default":false,"squash_merge_commit_message":"COMMIT_MESSAGES","squash_merge_commit_title":"COMMIT_OR_PR_TITLE","merge_commit_message":"PR_TITLE","merge_commit_title":"MERGE_MESSAGE","network_count":0,"subscribers_count":1} gunicorn-web stdout | 2023-08-08 06:19:43,157 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/keys/85931051 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:43,157 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/keys/85931051 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:43 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"b1465e50d9b7e6404e609fd5a12418002519bc58889f535f8b21ee258265a97f"', 'last-modified': 'Tue, 08 Aug 2023 06:19:34 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4916', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '84', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DA9E:0147:F334D7:1F2DBB9:64D1DE7F'} {"id":85931051,"key":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/MC598ROqhikoUsgPCEVjPQKyDXjDbMPPf1pSgM/YOSOI9zl48jBCJ25CJ77gRCfmZqEF7BQrlaWpy7kTvZdiT9/RrZF9U3BFF6NmhCuA86lGg9o624dnbiK0pTn1ajiofCdnr/ZJgVoY1Gcr++SapnqXzaG4/pUmygb9vk5AodBGhIL40bdN/4jZYfd5tEnfgKeTUC7umCymG69zZWVLNQfQ8H97Ggxr2jHrJDRRN7pLNYz2dLHXMe8FcMZ+uW5UT9y/CnlhlGBNRDbzOcd3pSrodk3FBLK9UXITxAzma40iEP8+ci74RJ4AQgjvnP2TfBdmtq5OXEgY0j7lLvaJ","url":"https://api.github.com/repos/hu-weihua/quay_build_test/keys/85931051","title":"Red Hat Quay Builder","verified":true,"created_at":"2023-08-08T06:19:34Z","read_only":false,"last_used":null,"added_by":"hu-weihua"} gunicorn-web stdout | 2023-08-08 06:19:43,308 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "DELETE /repos/hu-weihua/quay_build_test/keys/85931051 HTTP/1.1" 204 0 gunicorn-web stdout | 2023-08-08 06:19:43,308 [200] [DEBUG] [github.Requester] DELETE https://api.github.com/repos/hu-weihua/quay_build_test/keys/85931051 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 204 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:43 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': '', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4915', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '85', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'vary': 'Accept-Encoding, Accept, X-Requested-With', 'x-github-request-id': 'DA9E:0147:F334ED:1F2DBE7:64D1DE7F'} gunicorn-web stdout | 2023-08-08 06:19:43,429 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "GET /repos/hu-weihua/quay_build_test/hooks/427820277 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:19:43,430 [200] [DEBUG] [github.Requester] GET https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 200 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:43 GMT', 'content-type': 'application/json; charset=utf-8', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=60, s-maxage=60', 'vary': 'Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With', 'etag': 'W/"2aca6ecd3d7c803ca92bc624e9e0b85a35af55ab69603b0aa096f7e33c17cdd9"', 'last-modified': 'Tue, 08 Aug 2023 06:19:34 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:repo_hook, read:repo_hook, repo, write:repo_hook', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4914', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '86', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'content-encoding': 'gzip', 'x-github-request-id': 'DA9E:0147:F3351D:1F2DC55:64D1DE7F'} {"type":"Repository","id":427820277,"name":"web","active":true,"events":["push"],"config":{"content_type":"json","insecure_ssl":"0","url":"https://$token:O971568QK6MOX8J0I4R6DB3XWH3LI6LILN1YSO8YKV90SE8S49TJLPWH1CEO2QS6@quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/webhooks/push/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5"},"updated_at":"2023-08-08T06:19:34Z","created_at":"2023-08-08T06:19:34Z","url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277","test_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277/test","ping_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277/pings","deliveries_url":"https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277/deliveries","last_response":{"code":502,"status":"connection_error","message":"failed to connect to host"}} gunicorn-web stdout | 2023-08-08 06:19:43,555 [200] [DEBUG] [urllib3.connectionpool] https://api.github.com:443 "DELETE /repos/hu-weihua/quay_build_test/hooks/427820277 HTTP/1.1" 204 0 gunicorn-web stdout | 2023-08-08 06:19:43,555 [200] [DEBUG] [github.Requester] DELETE https://api.github.com/repos/hu-weihua/quay_build_test/hooks/427820277 {'Authorization': 'token (oauth token removed)', 'User-Agent': 'PyGithub/Python'} None ==> 204 {'server': 'GitHub.com', 'date': 'Tue, 08 Aug 2023 06:19:43 GMT', 'x-oauth-scopes': 'repo, user:email', 'x-accepted-oauth-scopes': 'admin:repo_hook, repo', 'x-oauth-client-id': '2ddf9640aa6b5e341a7d', 'x-github-media-type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'x-ratelimit-limit': '5000', 'x-ratelimit-remaining': '4913', 'x-ratelimit-reset': '1691477458', 'x-ratelimit-used': '87', 'x-ratelimit-resource': 'core', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'vary': 'Accept-Encoding, Accept, X-Requested-With', 'x-github-request-id': 'DA9E:0147:F33540:1F2DCAF:64D1DE7F'} gunicorn-web stdout | 2023-08-08 06:19:43,557 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:43,561 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:43,562 [200] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [23, 4, 3, 4, datetime.datetime(2023, 8, 8, 6, 19, 43, 560531), '10.130.2.1', '{"repo": "github", "trigger_id": "38fa6ab9-97e0-475c-bbf6-55d5541dc9b5", "service": "github"}']) gunicorn-web stdout | 2023-08-08 06:19:43,566 [200] [DEBUG] [peewee] ('UPDATE "repositorybuild" SET "trigger_id" = %s WHERE ("repositorybuild"."trigger_id" = %s)', [None, 5]) gunicorn-web stdout | 2023-08-08 06:19:43,567 [200] [DEBUG] [peewee] ('DELETE FROM "repositorybuildtrigger" WHERE ("repositorybuildtrigger"."id" = %s)', [5]) gunicorn-web stdout | 2023-08-08 06:19:43,571 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."friendly_name", "t1"."token_name", "t1"."token_code", "t1"."repository_id", "t1"."created", "t1"."role_id", "t1"."temporary", "t1"."kind_id" FROM "accesstoken" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [6, 1, 0]) gunicorn-web stdout | 2023-08-08 06:19:43,572 [200] [DEBUG] [peewee] ('DELETE FROM "accesstoken" WHERE ("accesstoken"."id" = %s)', [6]) gunicorn-web stdout | 2023-08-08 06:19:43,576 [200] [DEBUG] [app] Ending request: urn:request:786ff321-f251-4494-a94c-f7f2c1fb2f42 (/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:786ff321-f251-4494-a94c-f7f2c1fb2f42', 'remote_addr': '10.130.2.1', 'http_method': 'DELETE', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 'path': '/api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5', 'parameters': {}, 'json_body': {}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:19:43,577 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:19:43 +0000] "DELETE /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5 HTTP/2.0" 204 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.663 296 0.663) gunicorn-web stdout | 2023-08-08 06:19:43,577 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:19:43 +0000] "DELETE /api/v1/repository/user1_build_test/github/trigger/38fa6ab9-97e0-475c-bbf6-55d5541dc9b5 HTTP/1.0" 204 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" builder stdout | 2023-08-08 06:19:43,778 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:19:43,787 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:19:43,788 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:19:43,792 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:19:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:19:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:55.371682+00:00 (in 9.999474 seconds) notificationworker stdout | 2023-08-08 06:19:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:55 UTC)" (scheduled at 2023-08-08 06:19:45.371682+00:00) notificationworker stdout | 2023-08-08 06:19:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:19:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 45, 372810), True, datetime.datetime(2023, 8, 8, 6, 19, 45, 372810), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:19:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:19:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:19:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:19:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:19:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:52.293757+00:00 (in 5.418513 seconds) chunkcleanupworker stdout | 2023-08-08 06:19:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:24:46 UTC)" (scheduled at 2023-08-08 06:19:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:19:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 19, 46, 875712), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:19:46,882 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 19, 46, 875712), True, datetime.datetime(2023, 8, 8, 6, 19, 46, 875712), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:19:46,884 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 19, 46, 875712), True, datetime.datetime(2023, 8, 8, 6, 19, 46, 875712), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 19, 46, 875712), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:19:46,885 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:19:46,885 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:24:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:19:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:19:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:25.514341+00:00 (in 38.115285 seconds) exportactionlogsworker stdout | 2023-08-08 06:19:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:47 UTC)" (scheduled at 2023-08-08 06:19:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:19:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:19:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 47, 399566), True, datetime.datetime(2023, 8, 8, 6, 19, 47, 399566), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:19:47,409 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:19:47,409 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:19:47,409 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:19:48,571 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:19:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:19:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:59.417660+00:00 (in 9.999339 seconds) gcworker stdout | 2023-08-08 06:19:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:59 UTC)" (scheduled at 2023-08-08 06:19:49.417660+00:00) gcworker stdout | 2023-08-08 06:19:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265989418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:19:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:19:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:19:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:19:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:19:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:19:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:19:56.890741+00:00 (in 7.202854 seconds) securityworker stdout | 2023-08-08 06:19:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:19 UTC)" (scheduled at 2023-08-08 06:19:49.687363+00:00) securityworker stdout | 2023-08-08 06:19:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:19:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:19:49,694 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:19:49,695 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:19:49,701 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:19:49,701 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:19:49,829 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:19:50,885 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:19:51,344 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:19:52,179 [199] [DEBUG] [app] Starting request: urn:request:19d79563-1b3f-4bc6-9548-7ddc4156c776 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:19:52,179 [198] [DEBUG] [app] Starting request: urn:request:a4729b8b-3ec0-408d-8f1e-c19c722899d4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:19:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:19:52,184 [209] [DEBUG] [app] Starting request: urn:request:0628b9f6-5a7a-4ae5-af74-7a016cade449 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:19:52,185 [209] [DEBUG] [app] Ending request: urn:request:0628b9f6-5a7a-4ae5-af74-7a016cade449 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0628b9f6-5a7a-4ae5-af74-7a016cade449', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:19:52,185 [205] [DEBUG] [app] Starting request: urn:request:3e5b1cf4-3a72-4be8-8b27-1c1555b84948 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:19:52,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:19:52,185 [205] [DEBUG] [app] Ending request: urn:request:3e5b1cf4-3a72-4be8-8b27-1c1555b84948 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3e5b1cf4-3a72-4be8-8b27-1c1555b84948', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:19:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:19:52,186 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:52,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:19:52,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:19:52,191 [199] [DEBUG] [app] Starting request: urn:request:a1b27af8-d070-4b98-8079-62c75133f039 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:19:52,191 [200] [DEBUG] [app] Starting request: urn:request:70864ce1-4c23-44d4-97b0-fc219b82e938 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:19:52,191 [199] [DEBUG] [app] Ending request: urn:request:a1b27af8-d070-4b98-8079-62c75133f039 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a1b27af8-d070-4b98-8079-62c75133f039', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:19:52,191 [200] [DEBUG] [app] Ending request: urn:request:70864ce1-4c23-44d4-97b0-fc219b82e938 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:70864ce1-4c23-44d4-97b0-fc219b82e938', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:19:52,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:19:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:19:52,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:19:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:19:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:19:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:19:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:19:52,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:19:52,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:19:52,202 [199] [DEBUG] [app] Ending request: urn:request:19d79563-1b3f-4bc6-9548-7ddc4156c776 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:19d79563-1b3f-4bc6-9548-7ddc4156c776', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:19:52,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:19:52,202 [198] [DEBUG] [app] Ending request: urn:request:a4729b8b-3ec0-408d-8f1e-c19c722899d4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a4729b8b-3ec0-408d-8f1e-c19c722899d4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:19:52,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:19:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:19:52,203 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:19:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:19:52,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:19:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:19:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:19:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:19:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:02.293757+00:00 (in 9.999452 seconds) chunkcleanupworker stdout | 2023-08-08 06:19:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:02 UTC)" (scheduled at 2023-08-08 06:19:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:19:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:19:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 52, 294820), True, datetime.datetime(2023, 8, 8, 6, 19, 52, 294820), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:19:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:19:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:19:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:02 UTC)" executed successfully builder stdout | 2023-08-08 06:19:53,802 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:19:53,812 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:19:53,812 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:19:53,816 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:19:54,023 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:19:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:19:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:13.097375+00:00 (in 18.812767 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:19:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:54 UTC)" (scheduled at 2023-08-08 06:19:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:19:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:19:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:19:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:19:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:36.579841+00:00 (in 42.185649 seconds) namespacegcworker stdout | 2023-08-08 06:19:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:54 UTC)" (scheduled at 2023-08-08 06:19:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:19:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:19:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 54, 394606), True, datetime.datetime(2023, 8, 8, 6, 19, 54, 394606), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:19:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:19:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:19:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:20:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:19:55,199 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:19:55,203 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:19:55,205 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:19:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:19:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:02.178530+00:00 (in 6.806279 seconds) notificationworker stdout | 2023-08-08 06:19:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:05 UTC)" (scheduled at 2023-08-08 06:19:55.371682+00:00) notificationworker stdout | 2023-08-08 06:19:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:19:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 19, 55, 372810), True, datetime.datetime(2023, 8, 8, 6, 19, 55, 372810), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:19:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:19:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:19:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:19:55,839 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:19:56,179 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:19:56,354 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:19:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:19:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:19.687363+00:00 (in 22.795994 seconds) securityworker stdout | 2023-08-08 06:19:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:26 UTC)" (scheduled at 2023-08-08 06:19:56.890741+00:00) securityworker stdout | 2023-08-08 06:19:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:19:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:19:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:19:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:19:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:19:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:19:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:19:56,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:19:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:19:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:19:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 14, 56, 897391), 1, 2]) securityworker stdout | 2023-08-08 06:19:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 14, 56, 897391), 1, 2]) securityworker stdout | 2023-08-08 06:19:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:19:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:19:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:19:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:19:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:19:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:19:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:19:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:19:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:19:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:19:57,303 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:19:57,601 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:19:57,740 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:19:57,802 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:19:57,835 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:19:57,839 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:19:57,929 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:19:58,003 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} buildlogsarchiver stdout | 2023-08-08 06:19:59,399 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} gcworker stdout | 2023-08-08 06:19:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:19:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:09.417660+00:00 (in 9.999380 seconds) gcworker stdout | 2023-08-08 06:19:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:09 UTC)" (scheduled at 2023-08-08 06:19:59.417660+00:00) gcworker stdout | 2023-08-08 06:19:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690265999418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:19:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:19:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:19:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:09 UTC)" executed successfully builder stdout | 2023-08-08 06:19:59,526 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:20:00,589 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:20:01,439 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:20:01,578 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:20:02,001 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:20:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:20:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:05.371682+00:00 (in 3.192622 seconds) notificationworker stdout | 2023-08-08 06:20:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:02 UTC)" (scheduled at 2023-08-08 06:20:02.178530+00:00) notificationworker stdout | 2023-08-08 06:20:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:20:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:20:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:20:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:12.293757+00:00 (in 9.999477 seconds) chunkcleanupworker stdout | 2023-08-08 06:20:02,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:12 UTC)" (scheduled at 2023-08-08 06:20:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:20:02,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:20:02,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 2, 295221), True, datetime.datetime(2023, 8, 8, 6, 20, 2, 295221), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:20:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:20:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:20:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:12 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:20:02,498 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:20:02,803 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:20:03,130 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:20:03,500 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:20:03,503 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:20:03,506 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:20:03,509 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:20:03,511 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:20:03,524 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:20:03,530 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:20:03,533 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:20:03,537 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:20:03,827 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:20:03,836 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:20:03,836 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:20:03,840 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:20:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:20:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:09.793658+00:00 (in 5.605684 seconds) repositorygcworker stdout | 2023-08-08 06:20:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:04 UTC)" (scheduled at 2023-08-08 06:20:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:20:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:20:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:20:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:20:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:15.371682+00:00 (in 9.999472 seconds) notificationworker stdout | 2023-08-08 06:20:05,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:15 UTC)" (scheduled at 2023-08-08 06:20:05.371682+00:00) notificationworker stdout | 2023-08-08 06:20:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:20:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 5, 373184), True, datetime.datetime(2023, 8, 8, 6, 20, 5, 373184), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:20:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:20:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:20:07,179 [197] [DEBUG] [app] Starting request: urn:request:ebce611f-0fe1-4b15-848d-e3eeeebecac8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:20:07,179 [200] [DEBUG] [app] Starting request: urn:request:7c2eec0e-3027-48e9-9093-a169029b4511 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:20:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:20:07,185 [210] [DEBUG] [app] Starting request: urn:request:f5603c06-3280-4d8e-a11c-6387d0695905 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:20:07,185 [206] [DEBUG] [app] Starting request: urn:request:d2b6749c-d0f1-4412-90a1-3ceff60c25d0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:20:07,185 [210] [DEBUG] [app] Ending request: urn:request:f5603c06-3280-4d8e-a11c-6387d0695905 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f5603c06-3280-4d8e-a11c-6387d0695905', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:20:07,186 [206] [DEBUG] [app] Ending request: urn:request:d2b6749c-d0f1-4412-90a1-3ceff60c25d0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d2b6749c-d0f1-4412-90a1-3ceff60c25d0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:20:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:20:07,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:20:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:07,191 [198] [DEBUG] [app] Starting request: urn:request:767acaf3-71c9-4c0b-867a-4659688bc37e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:20:07,191 [198] [DEBUG] [app] Ending request: urn:request:767acaf3-71c9-4c0b-867a-4659688bc37e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:767acaf3-71c9-4c0b-867a-4659688bc37e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:20:07,192 [199] [DEBUG] [app] Starting request: urn:request:0d996221-b41e-47e0-9840-4e111f56342b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:20:07,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:20:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:07,192 [199] [DEBUG] [app] Ending request: urn:request:0d996221-b41e-47e0-9840-4e111f56342b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0d996221-b41e-47e0-9840-4e111f56342b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:20:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:20:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:20:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:20:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:20:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:20:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:20:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:37.197412+00:00 (in 29.999533 seconds) buildlogsarchiver stdout | 2023-08-08 06:20:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:37 UTC)" (scheduled at 2023-08-08 06:20:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:20:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 20, 7, 198466), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:20:07,202 [200] [DEBUG] [app] Ending request: urn:request:7c2eec0e-3027-48e9-9093-a169029b4511 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7c2eec0e-3027-48e9-9093-a169029b4511', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:20:07,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:20:07,203 [197] [DEBUG] [app] Ending request: urn:request:ebce611f-0fe1-4b15-848d-e3eeeebecac8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ebce611f-0fe1-4b15-848d-e3eeeebecac8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:20:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:20:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:20:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:20:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:20:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:20:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:20:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:20:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:20:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:20:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:20:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:20:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:20:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:20:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:19.417660+00:00 (in 9.999329 seconds) gcworker stdout | 2023-08-08 06:20:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:19 UTC)" (scheduled at 2023-08-08 06:20:09.417660+00:00) gcworker stdout | 2023-08-08 06:20:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266009418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:20:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:20:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:20:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:20:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:20:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:04.187429+00:00 (in 54.393203 seconds) repositorygcworker stdout | 2023-08-08 06:20:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:09 UTC)" (scheduled at 2023-08-08 06:20:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:20:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:20:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 9, 794824), True, datetime.datetime(2023, 8, 8, 6, 20, 9, 794824), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:20:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:20:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:20:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:20:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:20:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:22.293757+00:00 (in 9.999405 seconds) chunkcleanupworker stdout | 2023-08-08 06:20:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:22 UTC)" (scheduled at 2023-08-08 06:20:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:20:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:20:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 12, 294955), True, datetime.datetime(2023, 8, 8, 6, 20, 12, 294955), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:20:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:20:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:20:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:20:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:20:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:54.284093+00:00 (in 41.186154 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:20:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:13 UTC)" (scheduled at 2023-08-08 06:20:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:20:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:20:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 13, 98434), True, datetime.datetime(2023, 8, 8, 6, 20, 13, 98434), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:20:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:20:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:20:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:13 UTC)" executed successfully builder stdout | 2023-08-08 06:20:13,841 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:20:13,850 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:20:13,850 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:20:13,854 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:20:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:25.371682+00:00 (in 9.999438 seconds) notificationworker stdout | 2023-08-08 06:20:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:25 UTC)" (scheduled at 2023-08-08 06:20:15.371682+00:00) notificationworker stdout | 2023-08-08 06:20:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:20:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 15, 372816), True, datetime.datetime(2023, 8, 8, 6, 20, 15, 372816), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:20:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:20:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:20:18,601 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:20:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:20:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:29.417660+00:00 (in 9.999472 seconds) gcworker stdout | 2023-08-08 06:20:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:29 UTC)" (scheduled at 2023-08-08 06:20:19.417660+00:00) gcworker stdout | 2023-08-08 06:20:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266019418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:20:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:20:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:20:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:20:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:20:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:26.890741+00:00 (in 7.202789 seconds) securityworker stdout | 2023-08-08 06:20:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:49 UTC)" (scheduled at 2023-08-08 06:20:19.687363+00:00) securityworker stdout | 2023-08-08 06:20:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:20:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:20:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:20:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:20:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:20:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:20:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 15, 19, 692365), 1, 2]) securityworker stdout | 2023-08-08 06:20:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 15, 19, 692365), 1, 2]) securityworker stdout | 2023-08-08 06:20:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:20:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:20:19,850 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:20:20,898 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:20:21,381 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:20:22,179 [197] [DEBUG] [app] Starting request: urn:request:fcd1be6e-91fa-4075-8288-4ad4b14146a6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:20:22,179 [198] [DEBUG] [app] Starting request: urn:request:888feb0d-3fb4-40f1-997f-3555a897d7a7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:20:22,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:22,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:20:22,184 [203] [DEBUG] [app] Starting request: urn:request:b6b27bbd-0053-452c-b142-2436e8813179 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:20:22,185 [210] [DEBUG] [app] Starting request: urn:request:764ad87a-b591-4757-9fb8-8359a4d66024 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:20:22,185 [203] [DEBUG] [app] Ending request: urn:request:b6b27bbd-0053-452c-b142-2436e8813179 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b6b27bbd-0053-452c-b142-2436e8813179', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:20:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:20:22,185 [210] [DEBUG] [app] Ending request: urn:request:764ad87a-b591-4757-9fb8-8359a4d66024 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:764ad87a-b591-4757-9fb8-8359a4d66024', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:20:22,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:20:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:20:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:22,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:22,190 [199] [DEBUG] [app] Starting request: urn:request:57ae3356-4e90-4da9-9cad-fb404fd19a83 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:20:22,191 [199] [DEBUG] [app] Ending request: urn:request:57ae3356-4e90-4da9-9cad-fb404fd19a83 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:57ae3356-4e90-4da9-9cad-fb404fd19a83', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:20:22,191 [200] [DEBUG] [app] Starting request: urn:request:509e20a8-fd7a-4770-b9e1-711e6101a89b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:20:22,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:20:22,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:22,191 [200] [DEBUG] [app] Ending request: urn:request:509e20a8-fd7a-4770-b9e1-711e6101a89b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:509e20a8-fd7a-4770-b9e1-711e6101a89b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:20:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:20:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:22,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:20:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:20:22,196 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:20:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:20:22,202 [198] [DEBUG] [app] Ending request: urn:request:888feb0d-3fb4-40f1-997f-3555a897d7a7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:888feb0d-3fb4-40f1-997f-3555a897d7a7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:20:22,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:20:22,202 [197] [DEBUG] [app] Ending request: urn:request:fcd1be6e-91fa-4075-8288-4ad4b14146a6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fcd1be6e-91fa-4075-8288-4ad4b14146a6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:20:22,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:20:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:20:22,202 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:20:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:20:22,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:20:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:20:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:20:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:20:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:28.874906+00:00 (in 6.580564 seconds) chunkcleanupworker stdout | 2023-08-08 06:20:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:32 UTC)" (scheduled at 2023-08-08 06:20:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:20:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:20:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 22, 294994), True, datetime.datetime(2023, 8, 8, 6, 20, 22, 294994), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:20:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:20:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:20:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:32 UTC)" executed successfully builder stdout | 2023-08-08 06:20:23,858 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:20:23,867 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:20:23,867 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:20:23,872 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:20:24,058 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:20:25,208 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:20:25,212 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:20:25,215 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:20:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:20:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:35.371682+00:00 (in 9.999462 seconds) notificationworker stdout | 2023-08-08 06:20:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:35 UTC)" (scheduled at 2023-08-08 06:20:25.371682+00:00) notificationworker stdout | 2023-08-08 06:20:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:20:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 25, 372802), True, datetime.datetime(2023, 8, 8, 6, 20, 25, 372802), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:20:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:20:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:20:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:20:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:47.398551+00:00 (in 21.883684 seconds) exportactionlogsworker stdout | 2023-08-08 06:20:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:25 UTC)" (scheduled at 2023-08-08 06:20:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:20:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:20:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:20:25,853 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:20:26,216 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:20:26,391 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:20:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:20:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:49.687363+00:00 (in 22.796064 seconds) securityworker stdout | 2023-08-08 06:20:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:56 UTC)" (scheduled at 2023-08-08 06:20:26.890741+00:00) securityworker stdout | 2023-08-08 06:20:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:20:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:20:26,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:20:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:20:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:20:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:20:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:20:26,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:20:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:20:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:20:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 15, 26, 897414), 1, 2]) securityworker stdout | 2023-08-08 06:20:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:26,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 15, 26, 897414), 1, 2]) securityworker stdout | 2023-08-08 06:20:26,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:26,911 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:26,911 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:20:26,911 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:20:26,911 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:20:26,911 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:20:26,911 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:20:26,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:20:26,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:20:26,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:20:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:20:27,339 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:20:27,610 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:20:27,756 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:20:27,818 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:20:27,851 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:20:27,854 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:20:27,966 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:20:28,041 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:20:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:20:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:32.293757+00:00 (in 3.418340 seconds) chunkcleanupworker stdout | 2023-08-08 06:20:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:28 UTC)" (scheduled at 2023-08-08 06:20:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:20:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:20:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:20:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:20:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:39.417660+00:00 (in 9.999494 seconds) gcworker stdout | 2023-08-08 06:20:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:39 UTC)" (scheduled at 2023-08-08 06:20:29.417660+00:00) gcworker stdout | 2023-08-08 06:20:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266029418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:20:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:20:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:20:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:20:29,436 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:20:29,564 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:20:30,627 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:20:31,460 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:20:31,616 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:20:32,026 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:20:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:20:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:42.293757+00:00 (in 9.999371 seconds) chunkcleanupworker stdout | 2023-08-08 06:20:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:42 UTC)" (scheduled at 2023-08-08 06:20:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:20:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:20:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 32, 294895), True, datetime.datetime(2023, 8, 8, 6, 20, 32, 294895), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:20:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:20:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:20:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:42 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:20:32,536 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:20:32,837 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:20:33,167 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:20:33,510 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:20:33,513 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:20:33,516 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:20:33,522 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:20:33,527 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:20:33,535 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:20:33,540 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:20:33,543 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:20:33,549 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:20:33,882 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:20:33,891 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:20:33,891 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:20:33,895 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:20:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:45.371682+00:00 (in 9.999415 seconds) notificationworker stdout | 2023-08-08 06:20:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:45 UTC)" (scheduled at 2023-08-08 06:20:35.371682+00:00) notificationworker stdout | 2023-08-08 06:20:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:20:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 35, 372843), True, datetime.datetime(2023, 8, 8, 6, 20, 35, 372843), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:20:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:20:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:20:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:20:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:54.393627+00:00 (in 17.813217 seconds) namespacegcworker stdout | 2023-08-08 06:20:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:36 UTC)" (scheduled at 2023-08-08 06:20:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:20:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:20:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:20:37,178 [199] [DEBUG] [app] Starting request: urn:request:8b9c618a-330a-46a6-a78b-b0e1748a773d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:20:37,180 [200] [DEBUG] [app] Starting request: urn:request:f40ac3a6-888a-489a-9782-62af093f610e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:20:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:20:37,184 [208] [DEBUG] [app] Starting request: urn:request:0cba305b-6758-496f-bc70-a32bf728fb06 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:20:37,184 [208] [DEBUG] [app] Ending request: urn:request:0cba305b-6758-496f-bc70-a32bf728fb06 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0cba305b-6758-496f-bc70-a32bf728fb06', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:20:37,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:20:37,185 [209] [DEBUG] [app] Starting request: urn:request:3f266b07-bac0-4e3c-b8d9-de4e4ac63edc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:20:37,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:20:37,185 [209] [DEBUG] [app] Ending request: urn:request:3f266b07-bac0-4e3c-b8d9-de4e4ac63edc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3f266b07-bac0-4e3c-b8d9-de4e4ac63edc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:20:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:20:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:37,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:37,190 [198] [DEBUG] [app] Starting request: urn:request:b9aa50c6-103a-4f0e-9d93-a3a9ad3ca78e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:20:37,191 [197] [DEBUG] [app] Starting request: urn:request:1b30fcab-be1e-4a02-a6c3-c7e0530289c9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:20:37,191 [198] [DEBUG] [app] Ending request: urn:request:b9aa50c6-103a-4f0e-9d93-a3a9ad3ca78e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b9aa50c6-103a-4f0e-9d93-a3a9ad3ca78e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:20:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:20:37,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:37,191 [197] [DEBUG] [app] Ending request: urn:request:1b30fcab-be1e-4a02-a6c3-c7e0530289c9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1b30fcab-be1e-4a02-a6c3-c7e0530289c9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:20:37,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:20:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2023-08-08 06:20:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:20:37,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:20:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:20:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:20:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:07.197412+00:00 (in 29.999509 seconds) buildlogsarchiver stdout | 2023-08-08 06:20:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:07 UTC)" (scheduled at 2023-08-08 06:20:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:20:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 20, 37, 198338), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:20:37,202 [200] [DEBUG] [app] Ending request: urn:request:f40ac3a6-888a-489a-9782-62af093f610e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f40ac3a6-888a-489a-9782-62af093f610e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:20:37,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:20:37,202 [199] [DEBUG] [app] Ending request: urn:request:8b9c618a-330a-46a6-a78b-b0e1748a773d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8b9c618a-330a-46a6-a78b-b0e1748a773d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:20:37,202 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:20:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.023 47 0.024) gunicorn-web stdout | 2023-08-08 06:20:37,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:20:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:20:37,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:20:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:20:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:20:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:20:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:20:37,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:20:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:20:37,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:20:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:20:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:49.417660+00:00 (in 9.999450 seconds) gcworker stdout | 2023-08-08 06:20:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:49 UTC)" (scheduled at 2023-08-08 06:20:39.417660+00:00) gcworker stdout | 2023-08-08 06:20:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266039418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:20:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:20:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:20:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:20:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:20:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:52.293757+00:00 (in 9.999421 seconds) chunkcleanupworker stdout | 2023-08-08 06:20:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:52 UTC)" (scheduled at 2023-08-08 06:20:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:20:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:20:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 42, 294906), True, datetime.datetime(2023, 8, 8, 6, 20, 42, 294906), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:20:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:20:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:20:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:52 UTC)" executed successfully builder stdout | 2023-08-08 06:20:43,898 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:20:43,907 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:20:43,907 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:20:43,911 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:20:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:55.371682+00:00 (in 9.999415 seconds) notificationworker stdout | 2023-08-08 06:20:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:55 UTC)" (scheduled at 2023-08-08 06:20:45.371682+00:00) notificationworker stdout | 2023-08-08 06:20:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:20:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 45, 372764), True, datetime.datetime(2023, 8, 8, 6, 20, 45, 372764), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:20:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:20:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:20:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:20:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:25.514341+00:00 (in 38.115237 seconds) exportactionlogsworker stdout | 2023-08-08 06:20:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:47 UTC)" (scheduled at 2023-08-08 06:20:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:20:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:20:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 47, 399488), True, datetime.datetime(2023, 8, 8, 6, 20, 47, 399488), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:20:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:20:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:20:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:20:48,639 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:20:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:20:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:59.417660+00:00 (in 9.999469 seconds) gcworker stdout | 2023-08-08 06:20:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:59 UTC)" (scheduled at 2023-08-08 06:20:49.417660+00:00) gcworker stdout | 2023-08-08 06:20:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266049418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:20:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:20:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:20:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:20:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:20:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:20:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:20:56.890741+00:00 (in 7.202776 seconds) securityworker stdout | 2023-08-08 06:20:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:19 UTC)" (scheduled at 2023-08-08 06:20:49.687363+00:00) securityworker stdout | 2023-08-08 06:20:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:20:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:20:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:20:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:20:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:20:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:20:49,862 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:20:50,932 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:20:51,417 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:20:52,179 [198] [DEBUG] [app] Starting request: urn:request:9187dec6-fc03-4f49-8e95-ead0c24d9968 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:20:52,179 [200] [DEBUG] [app] Starting request: urn:request:9b87cefa-7f83-4a8c-be45-83da6caaa391 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:20:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:52,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:20:52,184 [207] [DEBUG] [app] Starting request: urn:request:44a8aa35-722b-4208-815c-784ffc292826 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:20:52,184 [209] [DEBUG] [app] Starting request: urn:request:c1cf156a-cc36-4ac3-a6a1-02a5ae6db6b2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:20:52,185 [207] [DEBUG] [app] Ending request: urn:request:44a8aa35-722b-4208-815c-784ffc292826 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:44a8aa35-722b-4208-815c-784ffc292826', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:20:52,185 [209] [DEBUG] [app] Ending request: urn:request:c1cf156a-cc36-4ac3-a6a1-02a5ae6db6b2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c1cf156a-cc36-4ac3-a6a1-02a5ae6db6b2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:20:52,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:20:52,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:20:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:52,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:20:52,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:20:52,191 [198] [DEBUG] [app] Starting request: urn:request:ef92dfdb-37d8-4f12-a845-f69e6ff9e1ef (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:20:52,191 [198] [DEBUG] [app] Ending request: urn:request:ef92dfdb-37d8-4f12-a845-f69e6ff9e1ef (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ef92dfdb-37d8-4f12-a845-f69e6ff9e1ef', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:20:52,191 [197] [DEBUG] [app] Starting request: urn:request:5ef55ca2-03ff-4481-b79b-f31349498a5b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:20:52,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:20:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:52,192 [197] [DEBUG] [app] Ending request: urn:request:5ef55ca2-03ff-4481-b79b-f31349498a5b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5ef55ca2-03ff-4481-b79b-f31349498a5b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:20:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:20:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:20:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:20:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:20:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:20:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:20:52,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:20:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:20:52,203 [198] [DEBUG] [app] Ending request: urn:request:9187dec6-fc03-4f49-8e95-ead0c24d9968 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9187dec6-fc03-4f49-8e95-ead0c24d9968', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:20:52,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:20:52,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:20:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:20:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:20:52,204 [200] [DEBUG] [app] Ending request: urn:request:9b87cefa-7f83-4a8c-be45-83da6caaa391 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9b87cefa-7f83-4a8c-be45-83da6caaa391', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:20:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:20:52,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:20:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:20:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:20:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:20:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:02.293757+00:00 (in 9.999485 seconds) chunkcleanupworker stdout | 2023-08-08 06:20:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:02 UTC)" (scheduled at 2023-08-08 06:20:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:20:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:20:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 52, 294787), True, datetime.datetime(2023, 8, 8, 6, 20, 52, 294787), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:20:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:20:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:20:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:02 UTC)" executed successfully builder stdout | 2023-08-08 06:20:53,921 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:20:53,930 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:20:53,930 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:20:53,936 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:20:54,087 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:20:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:20:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:13.097375+00:00 (in 18.812649 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:20:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:54 UTC)" (scheduled at 2023-08-08 06:20:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:20:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:20:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:20:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:20:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:36.579841+00:00 (in 42.185683 seconds) namespacegcworker stdout | 2023-08-08 06:20:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:54 UTC)" (scheduled at 2023-08-08 06:20:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:20:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:20:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 54, 394775), True, datetime.datetime(2023, 8, 8, 6, 20, 54, 394775), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:20:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:20:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:20:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:21:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:20:55,219 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:20:55,222 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:20:55,226 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:20:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:20:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:02.178530+00:00 (in 6.806294 seconds) notificationworker stdout | 2023-08-08 06:20:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:05 UTC)" (scheduled at 2023-08-08 06:20:55.371682+00:00) notificationworker stdout | 2023-08-08 06:20:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:20:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 20, 55, 372808), True, datetime.datetime(2023, 8, 8, 6, 20, 55, 372808), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:20:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:20:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:20:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:20:55,877 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:20:56,241 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:20:56,402 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:20:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:20:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:19.687363+00:00 (in 22.795970 seconds) securityworker stdout | 2023-08-08 06:20:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:26 UTC)" (scheduled at 2023-08-08 06:20:56.890741+00:00) securityworker stdout | 2023-08-08 06:20:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:20:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:20:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:20:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:20:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:20:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:20:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:20:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:20:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:20:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:20:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 15, 56, 896914), 1, 2]) securityworker stdout | 2023-08-08 06:20:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 15, 56, 896914), 1, 2]) securityworker stdout | 2023-08-08 06:20:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:20:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:20:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:20:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:20:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:20:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:20:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:20:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:20:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:20:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:20:57,377 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:20:57,619 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:20:57,768 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:20:57,833 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:20:57,864 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:20:57,869 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:20:57,975 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:20:58,079 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:20:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:20:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:09.417660+00:00 (in 9.999450 seconds) gcworker stdout | 2023-08-08 06:20:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:09 UTC)" (scheduled at 2023-08-08 06:20:59.417660+00:00) gcworker stdout | 2023-08-08 06:20:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266059418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:20:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:20:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:20:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:20:59,447 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:20:59,604 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:21:00,635 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:21:01,494 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:21:01,642 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:21:02,064 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:21:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:21:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:05.371682+00:00 (in 3.192595 seconds) notificationworker stdout | 2023-08-08 06:21:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:02 UTC)" (scheduled at 2023-08-08 06:21:02.178530+00:00) notificationworker stdout | 2023-08-08 06:21:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:21:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:21:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:21:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:12.293757+00:00 (in 9.999483 seconds) chunkcleanupworker stdout | 2023-08-08 06:21:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:12 UTC)" (scheduled at 2023-08-08 06:21:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:21:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:21:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 2, 294872), True, datetime.datetime(2023, 8, 8, 6, 21, 2, 294872), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:21:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:21:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:21:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:12 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:21:02,573 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:21:02,868 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:21:03,193 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:21:03,522 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:21:03,525 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:21:03,529 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:21:03,534 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:21:03,538 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:21:03,546 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:21:03,549 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:21:03,552 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:21:03,560 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:21:03,946 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:21:03,955 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:21:03,955 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:21:03,959 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:21:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:21:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:09.793658+00:00 (in 5.605692 seconds) repositorygcworker stdout | 2023-08-08 06:21:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:04 UTC)" (scheduled at 2023-08-08 06:21:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:21:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:21:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:21:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:21:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:15.371682+00:00 (in 9.999502 seconds) notificationworker stdout | 2023-08-08 06:21:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:15 UTC)" (scheduled at 2023-08-08 06:21:05.371682+00:00) notificationworker stdout | 2023-08-08 06:21:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:21:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 5, 372669), True, datetime.datetime(2023, 8, 8, 6, 21, 5, 372669), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:21:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:21:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:21:07,178 [199] [DEBUG] [app] Starting request: urn:request:af532fc6-4945-42e0-a688-5c095a87a53c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:21:07,178 [200] [DEBUG] [app] Starting request: urn:request:78b1609b-cde5-48a6-982b-a74217462ae8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:21:07,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:07,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:21:07,184 [210] [DEBUG] [app] Starting request: urn:request:2e65ff19-4c59-4944-ab36-d5409202d659 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:21:07,184 [203] [DEBUG] [app] Starting request: urn:request:348436cc-b470-4045-8293-0da4f0799af2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:21:07,185 [210] [DEBUG] [app] Ending request: urn:request:2e65ff19-4c59-4944-ab36-d5409202d659 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2e65ff19-4c59-4944-ab36-d5409202d659', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:21:07,185 [203] [DEBUG] [app] Ending request: urn:request:348436cc-b470-4045-8293-0da4f0799af2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:348436cc-b470-4045-8293-0da4f0799af2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:21:07,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:07,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:21:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:21:07,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:07,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:07,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:07,190 [199] [DEBUG] [app] Starting request: urn:request:38c49dd9-e83c-4940-8a94-0ad938ca66b4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:21:07,190 [199] [DEBUG] [app] Ending request: urn:request:38c49dd9-e83c-4940-8a94-0ad938ca66b4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:38c49dd9-e83c-4940-8a94-0ad938ca66b4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:21:07,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:07,191 [197] [DEBUG] [app] Starting request: urn:request:bd2a5603-4f6a-40ea-b8c5-078cd8f854c5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:21:07,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:07,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:21:07,192 [197] [DEBUG] [app] Ending request: urn:request:bd2a5603-4f6a-40ea-b8c5-078cd8f854c5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bd2a5603-4f6a-40ea-b8c5-078cd8f854c5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:21:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:21:07,196 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:21:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:21:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:21:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:37.197412+00:00 (in 29.999418 seconds) buildlogsarchiver stdout | 2023-08-08 06:21:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:37 UTC)" (scheduled at 2023-08-08 06:21:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:21:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 21, 7, 198681), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:21:07,202 [199] [DEBUG] [app] Ending request: urn:request:af532fc6-4945-42e0-a688-5c095a87a53c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:af532fc6-4945-42e0-a688-5c095a87a53c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:21:07,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:21:07,202 [200] [DEBUG] [app] Ending request: urn:request:78b1609b-cde5-48a6-982b-a74217462ae8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:78b1609b-cde5-48a6-982b-a74217462ae8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:21:07,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:21:07,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:21:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:21:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:21:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:21:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:21:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:21:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:21:07,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:21:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:21:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:21:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:21:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:21:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:19.417660+00:00 (in 9.999429 seconds) gcworker stdout | 2023-08-08 06:21:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:19 UTC)" (scheduled at 2023-08-08 06:21:09.417660+00:00) gcworker stdout | 2023-08-08 06:21:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266069418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:21:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:21:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:21:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:21:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:21:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:04.187429+00:00 (in 54.393235 seconds) repositorygcworker stdout | 2023-08-08 06:21:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:09 UTC)" (scheduled at 2023-08-08 06:21:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:21:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:21:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 9, 794697), True, datetime.datetime(2023, 8, 8, 6, 21, 9, 794697), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:21:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:21:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:21:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:21:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:21:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:22.293757+00:00 (in 9.999432 seconds) chunkcleanupworker stdout | 2023-08-08 06:21:12,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:22 UTC)" (scheduled at 2023-08-08 06:21:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:21:12,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:21:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 12, 295256), True, datetime.datetime(2023, 8, 8, 6, 21, 12, 295256), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:21:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:21:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:21:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:21:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:21:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:31.283850+00:00 (in 18.185893 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:21:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:13 UTC)" (scheduled at 2023-08-08 06:21:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:21:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:21:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 13, 98601), True, datetime.datetime(2023, 8, 8, 6, 21, 13, 98601), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:21:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:21:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:21:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:13 UTC)" executed successfully builder stdout | 2023-08-08 06:21:13,969 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:21:13,979 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:21:13,979 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:21:13,983 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:21:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:25.371682+00:00 (in 9.999473 seconds) notificationworker stdout | 2023-08-08 06:21:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:25 UTC)" (scheduled at 2023-08-08 06:21:15.371682+00:00) notificationworker stdout | 2023-08-08 06:21:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:21:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 15, 372791), True, datetime.datetime(2023, 8, 8, 6, 21, 15, 372791), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:21:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:21:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:21:18,667 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:21:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:21:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:29.417660+00:00 (in 9.999433 seconds) gcworker stdout | 2023-08-08 06:21:19,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:29 UTC)" (scheduled at 2023-08-08 06:21:19.417660+00:00) gcworker stdout | 2023-08-08 06:21:19,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266079419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:21:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:21:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:21:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:21:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:21:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:26.890741+00:00 (in 7.202788 seconds) securityworker stdout | 2023-08-08 06:21:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:49 UTC)" (scheduled at 2023-08-08 06:21:19.687363+00:00) securityworker stdout | 2023-08-08 06:21:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:21:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:21:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:21:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:21:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:21:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:21:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 16, 19, 692486), 1, 2]) securityworker stdout | 2023-08-08 06:21:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 16, 19, 692486), 1, 2]) securityworker stdout | 2023-08-08 06:21:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:21:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:21:19,899 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:21:20,940 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:21:21,452 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:21:22,178 [200] [DEBUG] [app] Starting request: urn:request:a972fbc6-3f53-4662-bb60-ee1c151c447e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:21:22,179 [199] [DEBUG] [app] Starting request: urn:request:3763b131-b401-432f-ba94-c64a6e9e1549 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:21:22,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:21:22,184 [203] [DEBUG] [app] Starting request: urn:request:35be5036-7555-414e-8903-396db2eeba89 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:21:22,184 [203] [DEBUG] [app] Ending request: urn:request:35be5036-7555-414e-8903-396db2eeba89 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:35be5036-7555-414e-8903-396db2eeba89', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:21:22,184 [205] [DEBUG] [app] Starting request: urn:request:74691897-9b71-4b3b-9c43-ee1d3d0be265 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:21:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:21:22,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:21:22,185 [205] [DEBUG] [app] Ending request: urn:request:74691897-9b71-4b3b-9c43-ee1d3d0be265 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:74691897-9b71-4b3b-9c43-ee1d3d0be265', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:21:22,185 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:22,186 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:22,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:22,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:22,190 [197] [DEBUG] [app] Starting request: urn:request:7d74dcc6-c89e-43c2-b057-6c9fe4980106 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:21:22,190 [197] [DEBUG] [app] Ending request: urn:request:7d74dcc6-c89e-43c2-b057-6c9fe4980106 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7d74dcc6-c89e-43c2-b057-6c9fe4980106', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:21:22,190 [198] [DEBUG] [app] Starting request: urn:request:d0eb3ac0-6ac5-4665-a125-f89fb0b7a8f8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:21:22,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:22,191 [198] [DEBUG] [app] Ending request: urn:request:d0eb3ac0-6ac5-4665-a125-f89fb0b7a8f8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d0eb3ac0-6ac5-4665-a125-f89fb0b7a8f8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:21:22,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:22,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:21:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:21:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:21:22,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:21:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:21:22,201 [200] [DEBUG] [app] Ending request: urn:request:a972fbc6-3f53-4662-bb60-ee1c151c447e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a972fbc6-3f53-4662-bb60-ee1c151c447e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:21:22,201 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:21:22,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:21:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:21:22,202 [199] [DEBUG] [app] Ending request: urn:request:3763b131-b401-432f-ba94-c64a6e9e1549 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3763b131-b401-432f-ba94-c64a6e9e1549', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:21:22,202 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:21:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:21:22,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:21:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:21:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:21:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:21:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:28.874906+00:00 (in 6.580562 seconds) chunkcleanupworker stdout | 2023-08-08 06:21:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:32 UTC)" (scheduled at 2023-08-08 06:21:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:21:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:21:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 22, 294902), True, datetime.datetime(2023, 8, 8, 6, 21, 22, 294902), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:21:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:21:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:21:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:32 UTC)" executed successfully builder stdout | 2023-08-08 06:21:23,994 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:21:24,003 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:21:24,003 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:21:24,007 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:21:24,124 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:21:25,231 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:21:25,234 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:21:25,237 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:21:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:21:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:35.371682+00:00 (in 9.999453 seconds) notificationworker stdout | 2023-08-08 06:21:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:35 UTC)" (scheduled at 2023-08-08 06:21:25.371682+00:00) notificationworker stdout | 2023-08-08 06:21:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:21:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 25, 372825), True, datetime.datetime(2023, 8, 8, 6, 21, 25, 372825), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:21:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:21:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:21:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:21:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:47.398551+00:00 (in 21.883695 seconds) exportactionlogsworker stdout | 2023-08-08 06:21:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:25 UTC)" (scheduled at 2023-08-08 06:21:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:21:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:21:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:21:25,915 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:21:26,274 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:21:26,413 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:21:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:21:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:49.687363+00:00 (in 22.796076 seconds) securityworker stdout | 2023-08-08 06:21:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:56 UTC)" (scheduled at 2023-08-08 06:21:26.890741+00:00) securityworker stdout | 2023-08-08 06:21:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:21:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:21:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:21:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:21:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:21:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:21:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:21:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:21:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:21:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:21:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 16, 26, 896739), 1, 2]) securityworker stdout | 2023-08-08 06:21:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 16, 26, 896739), 1, 2]) securityworker stdout | 2023-08-08 06:21:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:21:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:21:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:21:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:21:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:21:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:21:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:21:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:21:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:21:27,414 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:21:27,628 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:21:27,782 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:21:27,847 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:21:27,878 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:21:27,882 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:21:28,012 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:21:28,117 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:21:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:21:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:32.293757+00:00 (in 3.418195 seconds) chunkcleanupworker stdout | 2023-08-08 06:21:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:28 UTC)" (scheduled at 2023-08-08 06:21:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:21:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:21:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:21:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:21:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:39.417660+00:00 (in 9.999456 seconds) gcworker stdout | 2023-08-08 06:21:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:39 UTC)" (scheduled at 2023-08-08 06:21:29.417660+00:00) gcworker stdout | 2023-08-08 06:21:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266089418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:21:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:21:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:21:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:21:29,470 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:21:29,644 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:21:30,655 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} securityscanningnotificationworker stdout | 2023-08-08 06:21:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:21:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:54.284093+00:00 (in 22.999726 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:21:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:26:31 UTC)" (scheduled at 2023-08-08 06:21:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:21:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 21, 31, 284846), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:21:31,292 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 21, 31, 284846), True, datetime.datetime(2023, 8, 8, 6, 21, 31, 284846), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:21:31,294 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 21, 31, 284846), True, datetime.datetime(2023, 8, 8, 6, 21, 31, 284846), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 21, 31, 284846), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:21:31,295 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:21:31,295 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:26:31 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:21:31,532 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:21:31,681 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:21:32,074 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:21:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:21:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:42.293757+00:00 (in 9.999443 seconds) chunkcleanupworker stdout | 2023-08-08 06:21:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:42 UTC)" (scheduled at 2023-08-08 06:21:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:21:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:21:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 32, 294908), True, datetime.datetime(2023, 8, 8, 6, 21, 32, 294908), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:21:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:21:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:21:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:42 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:21:32,601 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:21:32,906 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:21:33,231 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:21:33,534 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:21:33,538 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:21:33,541 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:21:33,544 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:21:33,547 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:21:33,555 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:21:33,560 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:21:33,563 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:21:33,569 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:21:34,018 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:21:34,027 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:21:34,027 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:21:34,032 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:21:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:45.371682+00:00 (in 9.999489 seconds) notificationworker stdout | 2023-08-08 06:21:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:45 UTC)" (scheduled at 2023-08-08 06:21:35.371682+00:00) notificationworker stdout | 2023-08-08 06:21:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:21:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 35, 372873), True, datetime.datetime(2023, 8, 8, 6, 21, 35, 372873), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:21:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:21:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:21:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:21:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:54.393627+00:00 (in 17.813157 seconds) namespacegcworker stdout | 2023-08-08 06:21:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:36 UTC)" (scheduled at 2023-08-08 06:21:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:21:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:21:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:21:37,178 [198] [DEBUG] [app] Starting request: urn:request:e44ade72-4cd1-45fb-85be-3fce288a1622 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:21:37,179 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:37,180 [197] [DEBUG] [app] Starting request: urn:request:9332f185-dee4-4e12-8346-23ba3b16307c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:21:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:37,182 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:21:37,183 [210] [DEBUG] [app] Starting request: urn:request:694a8765-df90-4179-bac4-857e9f5c19a2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:21:37,184 [210] [DEBUG] [app] Ending request: urn:request:694a8765-df90-4179-bac4-857e9f5c19a2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:694a8765-df90-4179-bac4-857e9f5c19a2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:21:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:21:37,184 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:37,184 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:21:37,185 [210] [DEBUG] [app] Starting request: urn:request:38846e2c-5810-4885-bc12-9bd528bbe013 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:21:37,185 [210] [DEBUG] [app] Ending request: urn:request:38846e2c-5810-4885-bc12-9bd528bbe013 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:38846e2c-5810-4885-bc12-9bd528bbe013', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:21:37,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:21:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:37,186 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:37,188 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:37,190 [200] [DEBUG] [app] Starting request: urn:request:92b48dd0-cf7a-474e-9b3c-63d4ec28b611 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:21:37,190 [199] [DEBUG] [app] Starting request: urn:request:8a4fa0e7-ce00-43b6-9a2b-0325f9b7ffe1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:21:37,190 [200] [DEBUG] [app] Ending request: urn:request:92b48dd0-cf7a-474e-9b3c-63d4ec28b611 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:92b48dd0-cf7a-474e-9b3c-63d4ec28b611', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:21:37,191 [199] [DEBUG] [app] Ending request: urn:request:8a4fa0e7-ce00-43b6-9a2b-0325f9b7ffe1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8a4fa0e7-ce00-43b6-9a2b-0325f9b7ffe1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:21:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:37,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:37,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:21:37,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:37,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:21:37,191 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:21:37,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:21:37,196 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:21:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:21:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:07.197412+00:00 (in 29.999547 seconds) buildlogsarchiver stdout | 2023-08-08 06:21:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:07 UTC)" (scheduled at 2023-08-08 06:21:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:21:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 21, 37, 198312), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:21:37,201 [197] [DEBUG] [app] Ending request: urn:request:9332f185-dee4-4e12-8346-23ba3b16307c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9332f185-dee4-4e12-8346-23ba3b16307c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:21:37,201 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:21:37,202 [198] [DEBUG] [app] Ending request: urn:request:e44ade72-4cd1-45fb-85be-3fce288a1622 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e44ade72-4cd1-45fb-85be-3fce288a1622', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:21:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.023 47 0.023) gunicorn-web stdout | 2023-08-08 06:21:37,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:21:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:21:37,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:21:37,202 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:21:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:21:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:21:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:21:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:21:37,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:21:37,214 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:21:37,214 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:21:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:21:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:49.417660+00:00 (in 9.999473 seconds) gcworker stdout | 2023-08-08 06:21:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:49 UTC)" (scheduled at 2023-08-08 06:21:39.417660+00:00) gcworker stdout | 2023-08-08 06:21:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266099418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:21:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:21:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:21:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:21:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:21:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:52.293757+00:00 (in 9.999422 seconds) chunkcleanupworker stdout | 2023-08-08 06:21:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:52 UTC)" (scheduled at 2023-08-08 06:21:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:21:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:21:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 42, 294788), True, datetime.datetime(2023, 8, 8, 6, 21, 42, 294788), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:21:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:21:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:21:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:52 UTC)" executed successfully builder stdout | 2023-08-08 06:21:44,042 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:21:44,052 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:21:44,052 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:21:44,056 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:21:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:55.371682+00:00 (in 9.999482 seconds) notificationworker stdout | 2023-08-08 06:21:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:55 UTC)" (scheduled at 2023-08-08 06:21:45.371682+00:00) notificationworker stdout | 2023-08-08 06:21:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:21:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 45, 372595), True, datetime.datetime(2023, 8, 8, 6, 21, 45, 372595), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:21:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:21:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:21:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:21:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:25.514341+00:00 (in 38.115330 seconds) exportactionlogsworker stdout | 2023-08-08 06:21:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:47 UTC)" (scheduled at 2023-08-08 06:21:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:21:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:21:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 47, 399367), True, datetime.datetime(2023, 8, 8, 6, 21, 47, 399367), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:21:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:21:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:21:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:21:48,696 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:21:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:21:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:59.417660+00:00 (in 9.999444 seconds) gcworker stdout | 2023-08-08 06:21:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:59 UTC)" (scheduled at 2023-08-08 06:21:49.417660+00:00) gcworker stdout | 2023-08-08 06:21:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266109418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:21:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:21:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:21:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:21:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:21:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:21:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:21:56.890741+00:00 (in 7.202801 seconds) securityworker stdout | 2023-08-08 06:21:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:19 UTC)" (scheduled at 2023-08-08 06:21:49.687363+00:00) securityworker stdout | 2023-08-08 06:21:49,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:21:49,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:21:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:21:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:21:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:21:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:21:49,908 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:21:50,963 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:21:51,490 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:21:52,180 [197] [DEBUG] [app] Starting request: urn:request:9f58eccb-9a83-46d5-a10b-6f2d782a6203 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:21:52,180 [200] [DEBUG] [app] Starting request: urn:request:141d4965-e5c7-4e0c-af87-723edb7f2e44 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:21:52,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:52,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:52,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:52,187 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:21:52,188 [210] [DEBUG] [app] Starting request: urn:request:5746f7e7-b781-45f1-9381-2350e3474771 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:21:52,188 [205] [DEBUG] [app] Starting request: urn:request:f91fd52a-41d5-4ac2-a8a7-fdab0847b9e8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:21:52,188 [210] [DEBUG] [app] Ending request: urn:request:5746f7e7-b781-45f1-9381-2350e3474771 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5746f7e7-b781-45f1-9381-2350e3474771', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:21:52,189 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:21:52,189 [205] [DEBUG] [app] Ending request: urn:request:f91fd52a-41d5-4ac2-a8a7-fdab0847b9e8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f91fd52a-41d5-4ac2-a8a7-fdab0847b9e8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:21:52,189 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:21:52,189 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:52,189 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:52,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:52,191 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:21:52,193 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:52,193 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:21:52,194 [200] [DEBUG] [app] Starting request: urn:request:519756d7-250b-46ff-b3e1-cdcb35417c76 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:21:52,194 [199] [DEBUG] [app] Starting request: urn:request:9d45c7c3-0cb0-48f7-b748-f9a88aade535 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:21:52,194 [200] [DEBUG] [app] Ending request: urn:request:519756d7-250b-46ff-b3e1-cdcb35417c76 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:519756d7-250b-46ff-b3e1-cdcb35417c76', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:21:52,195 [199] [DEBUG] [app] Ending request: urn:request:9d45c7c3-0cb0-48f7-b748-f9a88aade535 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9d45c7c3-0cb0-48f7-b748-f9a88aade535', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:21:52,195 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:21:52,195 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:52,195 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:21:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:21:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:21:52,195 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:21:52,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:21:52,196 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:21:52,200 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:21:52,201 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:21:52,206 [200] [DEBUG] [app] Ending request: urn:request:141d4965-e5c7-4e0c-af87-723edb7f2e44 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:141d4965-e5c7-4e0c-af87-723edb7f2e44', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:21:52,206 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:21:52,206 [197] [DEBUG] [app] Ending request: urn:request:9f58eccb-9a83-46d5-a10b-6f2d782a6203 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9f58eccb-9a83-46d5-a10b-6f2d782a6203', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:21:52,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:21:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:21:52,206 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:21:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:21:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) gunicorn-web stdout | 2023-08-08 06:21:52,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:21:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:21:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:21:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:02.293757+00:00 (in 9.999380 seconds) chunkcleanupworker stdout | 2023-08-08 06:21:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:02 UTC)" (scheduled at 2023-08-08 06:21:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:21:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:21:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 52, 294883), True, datetime.datetime(2023, 8, 8, 6, 21, 52, 294883), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:21:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:21:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:21:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:02 UTC)" executed successfully builder stdout | 2023-08-08 06:21:54,067 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:21:54,076 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:21:54,076 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:21:54,080 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:21:54,132 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:21:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:21:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:13.097375+00:00 (in 18.812710 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:21:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:54 UTC)" (scheduled at 2023-08-08 06:21:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:21:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:21:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:21:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:21:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:36.579841+00:00 (in 42.185530 seconds) namespacegcworker stdout | 2023-08-08 06:21:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:54 UTC)" (scheduled at 2023-08-08 06:21:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:21:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:21:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 54, 394804), True, datetime.datetime(2023, 8, 8, 6, 21, 54, 394804), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:21:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:21:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:21:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:22:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:21:55,240 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:21:55,244 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:21:55,247 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:21:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:21:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:02.178530+00:00 (in 6.806323 seconds) notificationworker stdout | 2023-08-08 06:21:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:05 UTC)" (scheduled at 2023-08-08 06:21:55.371682+00:00) notificationworker stdout | 2023-08-08 06:21:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:21:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 21, 55, 372580), True, datetime.datetime(2023, 8, 8, 6, 21, 55, 372580), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:21:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:21:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:21:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:21:55,953 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:21:56,289 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:21:56,440 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:21:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:21:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:19.687363+00:00 (in 22.795940 seconds) securityworker stdout | 2023-08-08 06:21:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:26 UTC)" (scheduled at 2023-08-08 06:21:56.890741+00:00) securityworker stdout | 2023-08-08 06:21:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:21:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:21:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:21:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:21:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:21:56,894 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:21:56,894 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:21:56,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:21:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:21:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:21:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 16, 56, 897543), 1, 2]) securityworker stdout | 2023-08-08 06:21:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 16, 56, 897543), 1, 2]) securityworker stdout | 2023-08-08 06:21:56,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:21:56,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:21:56,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:21:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:21:56,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:21:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:21:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:21:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:21:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:21:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:21:57,423 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:21:57,638 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:21:57,795 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:21:57,863 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:21:57,891 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:21:57,896 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:21:58,049 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:21:58,155 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:21:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:21:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:09.417660+00:00 (in 9.999477 seconds) gcworker stdout | 2023-08-08 06:21:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:09 UTC)" (scheduled at 2023-08-08 06:21:59.417660+00:00) gcworker stdout | 2023-08-08 06:21:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266119418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:21:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:21:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:21:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:21:59,479 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:21:59,679 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:22:00,693 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:22:01,553 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:22:01,690 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:22:02,092 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:22:02,179 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:22:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:05.371682+00:00 (in 3.192372 seconds) notificationworker stdout | 2023-08-08 06:22:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:02 UTC)" (scheduled at 2023-08-08 06:22:02.178530+00:00) notificationworker stdout | 2023-08-08 06:22:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:22:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:22:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:22:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:12.293757+00:00 (in 9.999429 seconds) chunkcleanupworker stdout | 2023-08-08 06:22:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:12 UTC)" (scheduled at 2023-08-08 06:22:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:22:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:22:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 2, 294828), True, datetime.datetime(2023, 8, 8, 6, 22, 2, 294828), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:22:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:22:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:22:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:12 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:22:02,610 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:22:02,932 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:22:03,268 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:22:03,545 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:22:03,550 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:22:03,555 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:22:03,560 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:22:03,565 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:22:03,570 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:22:03,575 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:22:03,578 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:22:03,581 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:22:04,090 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:22:04,099 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:22:04,099 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:22:04,104 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:22:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:22:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:09.793658+00:00 (in 5.605703 seconds) repositorygcworker stdout | 2023-08-08 06:22:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:04 UTC)" (scheduled at 2023-08-08 06:22:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:22:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:22:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:22:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:22:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:15.371682+00:00 (in 9.999526 seconds) notificationworker stdout | 2023-08-08 06:22:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:15 UTC)" (scheduled at 2023-08-08 06:22:05.371682+00:00) notificationworker stdout | 2023-08-08 06:22:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:22:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 5, 372528), True, datetime.datetime(2023, 8, 8, 6, 22, 5, 372528), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:22:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:22:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:22:07,179 [200] [DEBUG] [app] Starting request: urn:request:53ad414e-7a6c-4b75-babb-a80a0b1acd33 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:22:07,179 [199] [DEBUG] [app] Starting request: urn:request:3661963b-0330-4ea4-bae1-ea46006e04a4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:22:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:07,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:22:07,185 [203] [DEBUG] [app] Starting request: urn:request:38a9f338-883e-498a-ada0-6d6d8a6aa661 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:22:07,186 [203] [DEBUG] [app] Ending request: urn:request:38a9f338-883e-498a-ada0-6d6d8a6aa661 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:38a9f338-883e-498a-ada0-6d6d8a6aa661', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:22:07,186 [209] [DEBUG] [app] Starting request: urn:request:e54773ea-b521-4349-858f-0fa68f24d0eb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:22:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:22:07,186 [209] [DEBUG] [app] Ending request: urn:request:e54773ea-b521-4349-858f-0fa68f24d0eb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e54773ea-b521-4349-858f-0fa68f24d0eb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:22:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:22:07,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:22:07,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:07,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:07,192 [197] [DEBUG] [app] Starting request: urn:request:32db78b0-cf82-4114-a542-620ef3e70442 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:22:07,192 [200] [DEBUG] [app] Starting request: urn:request:a0a943b7-e340-4a57-a958-6f0c6aad3a64 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:22:07,192 [197] [DEBUG] [app] Ending request: urn:request:32db78b0-cf82-4114-a542-620ef3e70442 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:32db78b0-cf82-4114-a542-620ef3e70442', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:22:07,192 [200] [DEBUG] [app] Ending request: urn:request:a0a943b7-e340-4a57-a958-6f0c6aad3a64 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a0a943b7-e340-4a57-a958-6f0c6aad3a64', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:22:07,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:22:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:22:07,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:22:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:07,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:22:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:22:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:22:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:37.197412+00:00 (in 29.999499 seconds) buildlogsarchiver stdout | 2023-08-08 06:22:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:37 UTC)" (scheduled at 2023-08-08 06:22:07.197412+00:00) gunicorn-web stdout | 2023-08-08 06:22:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:22:07,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:22:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 22, 7, 198467), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:22:07,204 [200] [DEBUG] [app] Ending request: urn:request:53ad414e-7a6c-4b75-babb-a80a0b1acd33 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:53ad414e-7a6c-4b75-babb-a80a0b1acd33', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:22:07,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:22:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:22:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:22:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:22:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:22:07,206 [199] [DEBUG] [app] Ending request: urn:request:3661963b-0330-4ea4-bae1-ea46006e04a4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3661963b-0330-4ea4-bae1-ea46006e04a4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:22:07,206 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:22:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) gunicorn-web stdout | 2023-08-08 06:22:07,206 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:22:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:22:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:22:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:22:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:22:07,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:22:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:22:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:19.417660+00:00 (in 9.999462 seconds) gcworker stdout | 2023-08-08 06:22:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:19 UTC)" (scheduled at 2023-08-08 06:22:09.417660+00:00) gcworker stdout | 2023-08-08 06:22:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266129418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:22:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:22:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:22:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:22:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:22:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:04.187429+00:00 (in 54.393257 seconds) repositorygcworker stdout | 2023-08-08 06:22:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:09 UTC)" (scheduled at 2023-08-08 06:22:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:22:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:22:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 9, 795127), True, datetime.datetime(2023, 8, 8, 6, 22, 9, 795127), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:22:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:22:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:22:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:22:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:22:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:22.293757+00:00 (in 9.999459 seconds) chunkcleanupworker stdout | 2023-08-08 06:22:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:22 UTC)" (scheduled at 2023-08-08 06:22:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:22:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:22:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 12, 294812), True, datetime.datetime(2023, 8, 8, 6, 22, 12, 294812), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:22:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:22:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:22:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:22:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:22:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:54.284093+00:00 (in 41.186212 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:22:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:13 UTC)" (scheduled at 2023-08-08 06:22:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:22:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:22:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 13, 98249), True, datetime.datetime(2023, 8, 8, 6, 22, 13, 98249), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:22:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:22:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:22:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:13 UTC)" executed successfully builder stdout | 2023-08-08 06:22:14,115 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:22:14,124 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:22:14,124 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:22:14,129 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:22:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:25.371682+00:00 (in 9.999487 seconds) notificationworker stdout | 2023-08-08 06:22:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:25 UTC)" (scheduled at 2023-08-08 06:22:15.371682+00:00) notificationworker stdout | 2023-08-08 06:22:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:22:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 15, 372554), True, datetime.datetime(2023, 8, 8, 6, 22, 15, 372554), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:22:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:22:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:22:18,729 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:22:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:22:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:29.417660+00:00 (in 9.999455 seconds) gcworker stdout | 2023-08-08 06:22:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:29 UTC)" (scheduled at 2023-08-08 06:22:19.417660+00:00) gcworker stdout | 2023-08-08 06:22:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266139418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:22:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:22:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:22:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:22:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:22:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:26.890741+00:00 (in 7.202775 seconds) securityworker stdout | 2023-08-08 06:22:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:49 UTC)" (scheduled at 2023-08-08 06:22:19.687363+00:00) securityworker stdout | 2023-08-08 06:22:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:22:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:22:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:22:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:22:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:22:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:22:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 17, 19, 692095), 1, 2]) securityworker stdout | 2023-08-08 06:22:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 17, 19, 692095), 1, 2]) securityworker stdout | 2023-08-08 06:22:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:22:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:22:19,937 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:22:20,979 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:22:21,499 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:22:22,179 [200] [DEBUG] [app] Starting request: urn:request:95924c52-88e8-4bac-8ae8-b0df1c912598 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:22:22,179 [197] [DEBUG] [app] Starting request: urn:request:d221506f-46b5-425b-bb9b-03db1c94b42c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:22:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:22:22,185 [203] [DEBUG] [app] Starting request: urn:request:532b2755-52bd-47d9-956e-28f2af6ffb90 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:22:22,185 [209] [DEBUG] [app] Starting request: urn:request:e01e0b12-334e-4cc8-ab7a-be3c77764414 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:22:22,185 [203] [DEBUG] [app] Ending request: urn:request:532b2755-52bd-47d9-956e-28f2af6ffb90 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:532b2755-52bd-47d9-956e-28f2af6ffb90', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:22:22,186 [209] [DEBUG] [app] Ending request: urn:request:e01e0b12-334e-4cc8-ab7a-be3c77764414 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e01e0b12-334e-4cc8-ab7a-be3c77764414', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:22:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:22:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:22:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:22:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:22,191 [197] [DEBUG] [app] Starting request: urn:request:8eba18b9-2cc8-4d8c-95f5-9c01b336694a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:22:22,192 [197] [DEBUG] [app] Ending request: urn:request:8eba18b9-2cc8-4d8c-95f5-9c01b336694a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8eba18b9-2cc8-4d8c-95f5-9c01b336694a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:22:22,192 [199] [DEBUG] [app] Starting request: urn:request:66b141ea-414d-4ea5-a72a-9d3a888a1392 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:22:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:22:22,192 [199] [DEBUG] [app] Ending request: urn:request:66b141ea-414d-4ea5-a72a-9d3a888a1392 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:66b141ea-414d-4ea5-a72a-9d3a888a1392', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:22:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:22:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:22:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:22:22,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:22:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:22:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:22:22,202 [197] [DEBUG] [app] Ending request: urn:request:d221506f-46b5-425b-bb9b-03db1c94b42c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d221506f-46b5-425b-bb9b-03db1c94b42c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:22:22,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:22:22,202 [200] [DEBUG] [app] Ending request: urn:request:95924c52-88e8-4bac-8ae8-b0df1c912598 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:95924c52-88e8-4bac-8ae8-b0df1c912598', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:22:22,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:22:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:22:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:22:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:22:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:22:22,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:22:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:22:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:22:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:28.874906+00:00 (in 6.580607 seconds) chunkcleanupworker stdout | 2023-08-08 06:22:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:32 UTC)" (scheduled at 2023-08-08 06:22:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:22:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:22:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 22, 294790), True, datetime.datetime(2023, 8, 8, 6, 22, 22, 294790), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:22:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:22:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:22:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:32 UTC)" executed successfully builder stdout | 2023-08-08 06:22:24,129 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:22:24,139 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:22:24,139 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:22:24,145 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:22:24,158 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:22:25,250 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:22:25,253 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:22:25,256 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:22:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:22:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:35.371682+00:00 (in 9.999471 seconds) notificationworker stdout | 2023-08-08 06:22:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:35 UTC)" (scheduled at 2023-08-08 06:22:25.371682+00:00) notificationworker stdout | 2023-08-08 06:22:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:22:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 25, 372572), True, datetime.datetime(2023, 8, 8, 6, 22, 25, 372572), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:22:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:22:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:22:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:22:25,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:47.398551+00:00 (in 21.883715 seconds) exportactionlogsworker stdout | 2023-08-08 06:22:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:25 UTC)" (scheduled at 2023-08-08 06:22:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:22:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:22:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:22:25,979 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:22:26,316 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:22:26,451 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:22:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:22:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:49.687363+00:00 (in 22.796101 seconds) securityworker stdout | 2023-08-08 06:22:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:56 UTC)" (scheduled at 2023-08-08 06:22:26.890741+00:00) securityworker stdout | 2023-08-08 06:22:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:22:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:22:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:22:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:22:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:22:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:22:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:22:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:22:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:22:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:22:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 17, 26, 897030), 1, 2]) securityworker stdout | 2023-08-08 06:22:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 17, 26, 897030), 1, 2]) securityworker stdout | 2023-08-08 06:22:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:22:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:22:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:22:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:22:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:22:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:22:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:22:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:22:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:22:27,447 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:22:27,649 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:22:27,808 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:22:27,878 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:22:27,904 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:22:27,911 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:22:28,057 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:22:28,193 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:22:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:22:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:32.293757+00:00 (in 3.418287 seconds) chunkcleanupworker stdout | 2023-08-08 06:22:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:28 UTC)" (scheduled at 2023-08-08 06:22:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:22:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:22:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:22:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:22:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:39.417660+00:00 (in 9.999450 seconds) gcworker stdout | 2023-08-08 06:22:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:39 UTC)" (scheduled at 2023-08-08 06:22:29.417660+00:00) gcworker stdout | 2023-08-08 06:22:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266149418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:22:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:22:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:22:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:22:29,499 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:22:29,717 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:22:30,701 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:22:31,591 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:22:31,728 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:22:32,126 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:22:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:22:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:42.293757+00:00 (in 9.999441 seconds) chunkcleanupworker stdout | 2023-08-08 06:22:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:42 UTC)" (scheduled at 2023-08-08 06:22:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:22:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:22:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 32, 294818), True, datetime.datetime(2023, 8, 8, 6, 22, 32, 294818), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:22:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:22:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:22:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:42 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:22:32,620 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:22:32,944 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:22:33,307 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:22:33,555 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:22:33,559 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:22:33,562 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:22:33,568 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:22:33,572 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:22:33,578 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:22:33,583 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:22:33,586 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:22:33,590 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:22:34,152 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:22:34,161 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:22:34,161 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:22:34,165 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:22:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:45.371682+00:00 (in 9.999458 seconds) notificationworker stdout | 2023-08-08 06:22:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:45 UTC)" (scheduled at 2023-08-08 06:22:35.371682+00:00) notificationworker stdout | 2023-08-08 06:22:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:22:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 35, 372662), True, datetime.datetime(2023, 8, 8, 6, 22, 35, 372662), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:22:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:22:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:22:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:22:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:54.393627+00:00 (in 17.813211 seconds) namespacegcworker stdout | 2023-08-08 06:22:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:36 UTC)" (scheduled at 2023-08-08 06:22:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:22:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:22:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:22:37,179 [200] [DEBUG] [app] Starting request: urn:request:00c66508-9797-4c13-aa81-4a187fdd058a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:22:37,179 [197] [DEBUG] [app] Starting request: urn:request:fc9ba8bd-ce2c-4e15-abf8-d37a5875cb13 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:22:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:22:37,185 [203] [DEBUG] [app] Starting request: urn:request:f09c6b95-4705-4989-b4e0-1f248c54960c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:22:37,186 [203] [DEBUG] [app] Ending request: urn:request:f09c6b95-4705-4989-b4e0-1f248c54960c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f09c6b95-4705-4989-b4e0-1f248c54960c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:22:37,186 [209] [DEBUG] [app] Starting request: urn:request:0ca51aac-b33c-412a-a033-b8a0c2edc98a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:22:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:22:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:22:37,187 [209] [DEBUG] [app] Ending request: urn:request:0ca51aac-b33c-412a-a033-b8a0c2edc98a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0ca51aac-b33c-412a-a033-b8a0c2edc98a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:22:37,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:22:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:37,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:37,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:37,192 [199] [DEBUG] [app] Starting request: urn:request:36ba3196-c002-447f-a536-351253df7df9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:22:37,192 [200] [DEBUG] [app] Starting request: urn:request:5e09b781-4f8a-4c11-9ceb-4d310b88bf43 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:22:37,193 [200] [DEBUG] [app] Ending request: urn:request:5e09b781-4f8a-4c11-9ceb-4d310b88bf43 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5e09b781-4f8a-4c11-9ceb-4d310b88bf43', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:22:37,193 [199] [DEBUG] [app] Ending request: urn:request:36ba3196-c002-447f-a536-351253df7df9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:36ba3196-c002-447f-a536-351253df7df9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:22:37,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:22:37,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2023-08-08 06:22:37,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:37,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:37,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:22:37,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:22:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:22:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:07.197412+00:00 (in 29.999494 seconds) buildlogsarchiver stdout | 2023-08-08 06:22:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:07 UTC)" (scheduled at 2023-08-08 06:22:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:22:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 22, 37, 198313), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:22:37,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:22:37,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:22:37,204 [200] [DEBUG] [app] Ending request: urn:request:00c66508-9797-4c13-aa81-4a187fdd058a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:00c66508-9797-4c13-aa81-4a187fdd058a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:22:37,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:22:37,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:22:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:22:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:22:37,204 [197] [DEBUG] [app] Ending request: urn:request:fc9ba8bd-ce2c-4e15-abf8-d37a5875cb13 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fc9ba8bd-ce2c-4e15-abf8-d37a5875cb13', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:22:37,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:22:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:22:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:22:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:22:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:22:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:22:37,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:22:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:22:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:22:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:22:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:49.417660+00:00 (in 9.999445 seconds) gcworker stdout | 2023-08-08 06:22:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:49 UTC)" (scheduled at 2023-08-08 06:22:39.417660+00:00) gcworker stdout | 2023-08-08 06:22:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266159418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:22:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:22:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:22:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:22:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:22:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:52.293757+00:00 (in 9.999440 seconds) chunkcleanupworker stdout | 2023-08-08 06:22:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:52 UTC)" (scheduled at 2023-08-08 06:22:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:22:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:22:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 42, 294871), True, datetime.datetime(2023, 8, 8, 6, 22, 42, 294871), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:22:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:22:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:22:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:52 UTC)" executed successfully builder stdout | 2023-08-08 06:22:44,175 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:22:44,184 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:22:44,185 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:22:44,189 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:22:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:55.371682+00:00 (in 9.999434 seconds) notificationworker stdout | 2023-08-08 06:22:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:55 UTC)" (scheduled at 2023-08-08 06:22:45.371682+00:00) notificationworker stdout | 2023-08-08 06:22:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:22:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 45, 372706), True, datetime.datetime(2023, 8, 8, 6, 22, 45, 372706), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:22:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:22:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:22:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:22:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:25.514341+00:00 (in 38.115297 seconds) exportactionlogsworker stdout | 2023-08-08 06:22:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:47 UTC)" (scheduled at 2023-08-08 06:22:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:22:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:22:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 47, 399470), True, datetime.datetime(2023, 8, 8, 6, 22, 47, 399470), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:22:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:22:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:22:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:22:48,746 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:22:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:22:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:59.417660+00:00 (in 9.999460 seconds) gcworker stdout | 2023-08-08 06:22:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:59 UTC)" (scheduled at 2023-08-08 06:22:49.417660+00:00) gcworker stdout | 2023-08-08 06:22:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266169418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:22:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:22:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:22:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:22:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:22:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:22:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:22:56.890741+00:00 (in 7.202849 seconds) securityworker stdout | 2023-08-08 06:22:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:19 UTC)" (scheduled at 2023-08-08 06:22:49.687363+00:00) securityworker stdout | 2023-08-08 06:22:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:22:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:22:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:22:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:22:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:22:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:22:49,975 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:22:51,016 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:22:51,537 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:22:52,179 [197] [DEBUG] [app] Starting request: urn:request:0b904523-8722-49dd-8db9-e36bf0669340 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:22:52,180 [199] [DEBUG] [app] Starting request: urn:request:0f161934-0d8f-48d5-88a3-07b50a5905eb (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:22:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:52,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:52,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:22:52,185 [203] [DEBUG] [app] Starting request: urn:request:653f6923-dc69-4ba0-a67f-3fb85792f703 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:22:52,186 [203] [DEBUG] [app] Ending request: urn:request:653f6923-dc69-4ba0-a67f-3fb85792f703 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:653f6923-dc69-4ba0-a67f-3fb85792f703', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:22:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:22:52,186 [208] [DEBUG] [app] Starting request: urn:request:11a38c44-75ff-4c6d-9ece-b413b1878530 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:22:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:22:52,187 [208] [DEBUG] [app] Ending request: urn:request:11a38c44-75ff-4c6d-9ece-b413b1878530 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:11a38c44-75ff-4c6d-9ece-b413b1878530', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:22:52,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:22:52,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:52,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:22:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:52,191 [199] [DEBUG] [app] Starting request: urn:request:8e0915bf-cb1d-4b11-bc39-57361b805804 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:22:52,192 [199] [DEBUG] [app] Ending request: urn:request:8e0915bf-cb1d-4b11-bc39-57361b805804 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8e0915bf-cb1d-4b11-bc39-57361b805804', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:22:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:22:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:22:52,193 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:22:52,194 [200] [DEBUG] [app] Starting request: urn:request:2cac649b-3eae-474b-9f71-ef2374fc6383 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:22:52,195 [200] [DEBUG] [app] Ending request: urn:request:2cac649b-3eae-474b-9f71-ef2374fc6383 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2cac649b-3eae-474b-9f71-ef2374fc6383', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:22:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:22:52,195 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:22:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:22:52,195 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:22:52,196 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:22:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:22:52,200 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:22:52,203 [197] [DEBUG] [app] Ending request: urn:request:0b904523-8722-49dd-8db9-e36bf0669340 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0b904523-8722-49dd-8db9-e36bf0669340', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:22:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:22:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:22:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:22:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:22:52,205 [199] [DEBUG] [app] Ending request: urn:request:0f161934-0d8f-48d5-88a3-07b50a5905eb (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0f161934-0d8f-48d5-88a3-07b50a5905eb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:22:52,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:22:52,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:22:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:22:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:22:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:22:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:02.293757+00:00 (in 9.999440 seconds) chunkcleanupworker stdout | 2023-08-08 06:22:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:02 UTC)" (scheduled at 2023-08-08 06:22:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:22:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:22:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 52, 294868), True, datetime.datetime(2023, 8, 8, 6, 22, 52, 294868), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:22:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:22:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:22:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:22:54,166 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:22:54,199 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:22:54,208 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:22:54,208 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:22:54,213 [55] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:22:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:22:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:13.097375+00:00 (in 18.812728 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:22:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:54 UTC)" (scheduled at 2023-08-08 06:22:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:22:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:22:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:22:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:22:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:36.579841+00:00 (in 42.185579 seconds) namespacegcworker stdout | 2023-08-08 06:22:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:54 UTC)" (scheduled at 2023-08-08 06:22:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:22:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:22:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 54, 394734), True, datetime.datetime(2023, 8, 8, 6, 22, 54, 394734), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:22:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:22:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:22:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:23:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:22:55,261 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:22:55,263 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:22:55,266 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:22:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:22:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:02.178530+00:00 (in 6.806279 seconds) notificationworker stdout | 2023-08-08 06:22:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:05 UTC)" (scheduled at 2023-08-08 06:22:55.371682+00:00) notificationworker stdout | 2023-08-08 06:22:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:22:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 22, 55, 372746), True, datetime.datetime(2023, 8, 8, 6, 22, 55, 372746), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:22:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:22:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:22:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:22:56,017 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:22:56,352 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:22:56,489 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:22:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:22:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:19.687363+00:00 (in 22.795715 seconds) securityworker stdout | 2023-08-08 06:22:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:26 UTC)" (scheduled at 2023-08-08 06:22:56.890741+00:00) securityworker stdout | 2023-08-08 06:22:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:22:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:22:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:22:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:22:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:22:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:22:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:22:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:22:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:22:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:22:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 17, 56, 897575), 1, 2]) securityworker stdout | 2023-08-08 06:22:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 17, 56, 897575), 1, 2]) securityworker stdout | 2023-08-08 06:22:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:22:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:22:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:22:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:22:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:22:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:22:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:22:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:22:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:22:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:22:57,484 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:22:57,657 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:22:57,823 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:22:57,893 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:22:57,918 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:22:57,927 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:22:58,084 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:22:58,230 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:22:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:22:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:09.417660+00:00 (in 9.999475 seconds) gcworker stdout | 2023-08-08 06:22:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:09 UTC)" (scheduled at 2023-08-08 06:22:59.417660+00:00) gcworker stdout | 2023-08-08 06:22:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266179418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:22:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:22:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:22:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:22:59,530 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:22:59,755 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:23:00,739 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:23:01,630 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:23:01,756 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:23:02,163 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} notificationworker stdout | 2023-08-08 06:23:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:23:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:05.371682+00:00 (in 3.192669 seconds) notificationworker stdout | 2023-08-08 06:23:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:02 UTC)" (scheduled at 2023-08-08 06:23:02.178530+00:00) notificationworker stdout | 2023-08-08 06:23:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:23:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:23:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:23:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:12.293757+00:00 (in 9.999426 seconds) chunkcleanupworker stdout | 2023-08-08 06:23:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:12 UTC)" (scheduled at 2023-08-08 06:23:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:23:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:23:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 2, 294861), True, datetime.datetime(2023, 8, 8, 6, 23, 2, 294861), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:23:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:23:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:23:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:12 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:23:02,631 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:23:02,980 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:23:03,341 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:23:03,564 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:23:03,570 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:23:03,573 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:23:03,576 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:23:03,579 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:23:03,588 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:23:03,594 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:23:03,597 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:23:03,601 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:23:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:23:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:09.793658+00:00 (in 5.605692 seconds) repositorygcworker stdout | 2023-08-08 06:23:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:04 UTC)" (scheduled at 2023-08-08 06:23:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:23:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:23:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:04 UTC)" executed successfully builder stdout | 2023-08-08 06:23:04,223 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:23:04,231 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:23:04,231 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:23:04,235 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:23:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:15.371682+00:00 (in 9.999487 seconds) notificationworker stdout | 2023-08-08 06:23:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:15 UTC)" (scheduled at 2023-08-08 06:23:05.371682+00:00) notificationworker stdout | 2023-08-08 06:23:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:23:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 5, 372746), True, datetime.datetime(2023, 8, 8, 6, 23, 5, 372746), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:23:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:23:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:23:06,512 [197] [DEBUG] [app] Starting request: urn:request:405d154e-cedd-4877-be30-1869c77c10d2 (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:23:06,512 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:06,512 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:06,513 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:06,520 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:23:06,521 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:23:06,521 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:06,521 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:23:06,521 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:06,521 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:06,521 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:06,521 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:06,524 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:23:06,530 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:23:06,531 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:06,536 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:23:06,541 [197] [DEBUG] [app] Ending request: urn:request:405d154e-cedd-4877-be30-1869c77c10d2 (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:405d154e-cedd-4877-be30-1869c77c10d2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=3', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:23:06,542 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:23:06 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.032 877 0.031) gunicorn-web stdout | 2023-08-08 06:23:06,542 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:23:06 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:23:07,179 [199] [DEBUG] [app] Starting request: urn:request:be7e0065-12d7-40b0-8836-f84a5af828e3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:23:07,179 [197] [DEBUG] [app] Starting request: urn:request:6276028c-91a0-41c5-ae35-be2c84fb7aae (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:23:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:23:07,185 [210] [DEBUG] [app] Starting request: urn:request:a061d535-5798-4803-a874-98835698a2d1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:23:07,185 [203] [DEBUG] [app] Starting request: urn:request:538c3cf6-73f4-4010-9548-342662f211c9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:23:07,185 [210] [DEBUG] [app] Ending request: urn:request:a061d535-5798-4803-a874-98835698a2d1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a061d535-5798-4803-a874-98835698a2d1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:23:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:23:07,186 [203] [DEBUG] [app] Ending request: urn:request:538c3cf6-73f4-4010-9548-342662f211c9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:538c3cf6-73f4-4010-9548-342662f211c9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:23:07,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:23:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:23:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:07,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:07,191 [200] [DEBUG] [app] Starting request: urn:request:af0f988b-33eb-4258-988d-69c603b61342 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:23:07,191 [199] [DEBUG] [app] Starting request: urn:request:22815664-cfb7-4774-a5e4-07a7d958bdbe (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:23:07,192 [200] [DEBUG] [app] Ending request: urn:request:af0f988b-33eb-4258-988d-69c603b61342 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:af0f988b-33eb-4258-988d-69c603b61342', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:23:07,192 [199] [DEBUG] [app] Ending request: urn:request:22815664-cfb7-4774-a5e4-07a7d958bdbe (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:22815664-cfb7-4774-a5e4-07a7d958bdbe', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:23:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:23:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:23:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:07,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:23:07,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:23:07,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:23:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:23:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:23:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:37.197412+00:00 (in 29.999547 seconds) buildlogsarchiver stdout | 2023-08-08 06:23:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:37 UTC)" (scheduled at 2023-08-08 06:23:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:23:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 23, 7, 198428), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:07,202 [199] [DEBUG] [app] Ending request: urn:request:be7e0065-12d7-40b0-8836-f84a5af828e3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:be7e0065-12d7-40b0-8836-f84a5af828e3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:23:07,202 [197] [DEBUG] [app] Ending request: urn:request:6276028c-91a0-41c5-ae35-be2c84fb7aae (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6276028c-91a0-41c5-ae35-be2c84fb7aae', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:23:07,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:23:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:23:07,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:23:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:23:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:23:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:23:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:23:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:23:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:23:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:23:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:23:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:23:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:37 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:23:08,717 [197] [DEBUG] [app] Starting request: urn:request:ef1382c4-c847-4691-82c4-4310482c4d10 (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:23:08,718 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:08,718 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:08,718 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:08,725 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:23:08,725 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:23:08,725 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:08,725 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:23:08,725 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:08,725 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:08,725 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:08,725 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:08,728 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:23:08,734 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:23:08,735 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:08,739 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:23:08,743 [197] [DEBUG] [app] Ending request: urn:request:ef1382c4-c847-4691-82c4-4310482c4d10 (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:ef1382c4-c847-4691-82c4-4310482c4d10', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=3', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:23:08,743 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:23:08,744 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:23:08 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:23:08 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 419 0.027) gcworker stdout | 2023-08-08 06:23:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:23:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:19.417660+00:00 (in 9.999467 seconds) gcworker stdout | 2023-08-08 06:23:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:19 UTC)" (scheduled at 2023-08-08 06:23:09.417660+00:00) gcworker stdout | 2023-08-08 06:23:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266189418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:23:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:23:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:23:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:19 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:23:09,694 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2023-08-08 06:23:09,694 [62] [INFO] [apscheduler.executors.default] Running job "GlobalPrometheusStatsWorker._try_report_stats (trigger: interval[1:00:00], next run at: 2023-08-08 06:23:09 UTC)" (scheduled at 2023-08-08 06:23:09.693962+00:00) globalpromstats stdout | 2023-08-08 06:23:09,695 [62] [DEBUG] [__main__] Attempting to report stats globalpromstats stdout | 2023-08-08 06:23:09,695 [62] [DEBUG] [util.locking] Acquiring global lock GLOBAL_PROM_STATS globalpromstats stdout | 2023-08-08 06:23:09,695 [62] [DEBUG] [redis_lock.acquire] Getting 'lock:GLOBAL_PROM_STATS' ... globalpromstats stdout | 2023-08-08 06:23:09,695 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 07:23:09.693962+00:00 (in 3599.999598 seconds) globalpromstats stdout | 2023-08-08 06:23:09,696 [62] [INFO] [redis_lock.acquire] Got lock for 'lock:GLOBAL_PROM_STATS'. globalpromstats stdout | 2023-08-08 06:23:09,696 [62] [DEBUG] [util.locking] Acquired lock GLOBAL_PROM_STATS globalpromstats stdout | 2023-08-08 06:23:09,696 [62] [DEBUG] [__main__] Reporting global stats globalpromstats stdout | 2023-08-08 06:23:09,696 [62] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "repository" AS "t1") AS "_wrapped"', []) globalpromstats stdout | 2023-08-08 06:23:09,703 [62] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ((("t1"."organization" = %s) AND ("t1"."robot" = %s)) AND ("t1"."id" NOT IN (SELECT "t2"."namespace_id" FROM "deletednamespace" AS "t2")))) AS "_wrapped"', [False, False]) globalpromstats stdout | 2023-08-08 06:23:09,705 [62] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ((("t1"."organization" = %s) AND ("t1"."robot" = %s)) AND ("t1"."enabled" = %s))) AS "_wrapped"', [True, False, True]) globalpromstats stdout | 2023-08-08 06:23:09,706 [62] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "robotaccounttoken" AS "t1") AS "_wrapped"', []) globalpromstats stdout | 2023-08-08 06:23:09,707 [62] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2023-08-08 06:23:09,708 [62] [DEBUG] [util.locking] Releasing lock GLOBAL_PROM_STATS globalpromstats stdout | 2023-08-08 06:23:09,708 [62] [DEBUG] [redis_lock.release] Releasing 'lock:GLOBAL_PROM_STATS'. globalpromstats stdout | 2023-08-08 06:23:09,708 [62] [DEBUG] [util.locking] Released lock GLOBAL_PROM_STATS globalpromstats stdout | 2023-08-08 06:23:09,708 [62] [INFO] [apscheduler.executors.default] Job "GlobalPrometheusStatsWorker._try_report_stats (trigger: interval[1:00:00], next run at: 2023-08-08 07:23:09 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:23:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:23:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:04.187429+00:00 (in 54.393267 seconds) repositorygcworker stdout | 2023-08-08 06:23:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:09 UTC)" (scheduled at 2023-08-08 06:23:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:23:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:23:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 9, 794708), True, datetime.datetime(2023, 8, 8, 6, 23, 9, 794708), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:23:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:23:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:23:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:23:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:23:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:22.293757+00:00 (in 9.999401 seconds) chunkcleanupworker stdout | 2023-08-08 06:23:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:22 UTC)" (scheduled at 2023-08-08 06:23:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:23:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:23:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 12, 294991), True, datetime.datetime(2023, 8, 8, 6, 23, 12, 294991), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:23:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:23:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:23:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:23:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:23:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:54.284093+00:00 (in 41.186142 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:23:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:13 UTC)" (scheduled at 2023-08-08 06:23:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:23:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:23:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 13, 98522), True, datetime.datetime(2023, 8, 8, 6, 23, 13, 98522), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:23:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:23:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:23:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:13 UTC)" executed successfully builder stdout | 2023-08-08 06:23:14,241 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:23:14,251 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:23:14,251 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:23:14,255 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:23:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:25.371682+00:00 (in 9.999447 seconds) notificationworker stdout | 2023-08-08 06:23:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:25 UTC)" (scheduled at 2023-08-08 06:23:15.371682+00:00) notificationworker stdout | 2023-08-08 06:23:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:23:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 15, 372843), True, datetime.datetime(2023, 8, 8, 6, 23, 15, 372843), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:23:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:23:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:23:15,490 [199] [DEBUG] [app] Starting request: urn:request:2eb757ae-9f75-45b6-a676-c44b98f14455 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:23:15,490 [197] [DEBUG] [app] Starting request: urn:request:3460a945-ed1f-4d92-a701-e31790b22b14 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:23:15,490 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,490 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,490 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,490 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,491 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,491 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,491 [198] [DEBUG] [app] Starting request: urn:request:d8e2e4e8-c573-4fa3-a476-c92f8d5d2ed0 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:23:15,491 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,491 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,492 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,497 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:23:15,498 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:23:15,498 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,498 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:23:15,498 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:23:15,498 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,498 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:23:15,498 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:23:15,498 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,498 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,498 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:15,498 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:15,498 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:15,498 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:23:15,498 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:15,498 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:15,498 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:23:15,499 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1', 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,499 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1', 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,499 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:23:15,501 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2023-08-08 06:23:15,502 [197] [DEBUG] [peewee] ('((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t3"."user_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'user1_build_test', 3, 3, 1, 'user1_build_test', 3, 3, 1, 'user1_build_test', 1, 3, 101]) gunicorn-web stdout | 2023-08-08 06:23:15,502 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:23:15,503 [198] [DEBUG] [peewee] ('((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t3"."user_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'user1', 3, 3, 1, 'user1', 3, 3, 1, 'user1', 1, 3, 101]) gunicorn-web stdout | 2023-08-08 06:23:15,504 [199] [DEBUG] [app] Ending request: urn:request:2eb757ae-9f75-45b6-a676-c44b98f14455 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:2eb757ae-9f75-45b6-a676-c44b98f14455', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:23:15,505 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:23:15 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 861 0.018) gunicorn-web stdout | 2023-08-08 06:23:15,505 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:23:15 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:23:15,507 [197] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Max("t1"."lifetime_start_ms") FROM "tag" AS "t1" WHERE ((("t1"."repository_id" IN (%s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) GROUP BY "t1"."repository_id"', [4, None, 1691475795507, False]) gunicorn-web stdout | 2023-08-08 06:23:15,508 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:23:15,509 [197] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Sum("t1"."count") FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" IN (%s)) AND ("t1"."date" >= %s)) GROUP BY "t1"."repository_id"', [4, datetime.date(2023, 8, 1)]) gunicorn-web stdout | 2023-08-08 06:23:15,510 [198] [DEBUG] [app] Ending request: urn:request:d8e2e4e8-c573-4fa3-a476-c92f8d5d2ed0 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:d8e2e4e8-c573-4fa3-a476-c92f8d5d2ed0', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'user1', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:23:15,511 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:23:15,511 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:23:15 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 101 0.022) gunicorn-web stdout | 2023-08-08 06:23:15,511 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:23:15 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:23:15,513 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,515 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:23:15,516 [197] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,517 [197] [DEBUG] [peewee] ('SELECT Sum("t1"."layers_compressed_size") AS "size_bytes" FROM "manifest" AS "t1" INNER JOIN (SELECT "t2"."manifest_id" FROM "tag" AS "t2" WHERE ((("t2"."hidden" = %s) AND (("t2"."lifetime_end_ms" IS %s) OR ("t2"."lifetime_end_ms" > %s))) AND ("t2"."repository_id" = %s)) GROUP BY "t2"."manifest_id" HAVING (Count("t2"."name") > %s)) AS "t3" ON ("t3"."manifest_id" = "t1"."id") WHERE ("t1"."repository_id" = %s)', [False, None, 1691475795517, 4, 0, 4]) gunicorn-web stdout | 2023-08-08 06:23:15,519 [197] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,520 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") VALUES (%s, %s) RETURNING "repositorysize"."id"', [4, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,523 [197] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:15,524 [197] [DEBUG] [app] Ending request: urn:request:3460a945-ed1f-4d92-a701-e31790b22b14 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:3460a945-ed1f-4d92-a701-e31790b22b14', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=false"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'user1_build_test', 'popularity': 'true', 'public': 'false', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:23:15,525 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:23:15 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=false"a=true&state=true HTTP/2.0" 200 280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.037 118 0.037) gunicorn-web stdout | 2023-08-08 06:23:15,525 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:23:15 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=false"a=true&state=true HTTP/1.0" 200 280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:23:18,784 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:23:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:23:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:29.417660+00:00 (in 9.999437 seconds) gcworker stdout | 2023-08-08 06:23:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:29 UTC)" (scheduled at 2023-08-08 06:23:19.417660+00:00) gcworker stdout | 2023-08-08 06:23:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266199418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:23:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:23:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:23:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:23:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:23:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:26.890741+00:00 (in 7.202866 seconds) securityworker stdout | 2023-08-08 06:23:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:49 UTC)" (scheduled at 2023-08-08 06:23:19.687363+00:00) securityworker stdout | 2023-08-08 06:23:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:23:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:23:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:23:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:23:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:23:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:23:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 18, 19, 692077), 1, 2]) securityworker stdout | 2023-08-08 06:23:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 18, 19, 692077), 1, 2]) securityworker stdout | 2023-08-08 06:23:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:23:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:23:20,008 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:23:21,046 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:23:21,567 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:23:22,178 [197] [DEBUG] [app] Starting request: urn:request:c97c843f-4067-4cc9-b4e0-043ad637eb17 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:23:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:22,180 [199] [DEBUG] [app] Starting request: urn:request:e43c9755-6866-45dd-a27e-317f8996a55e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:23:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:22,182 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:23:22,184 [206] [DEBUG] [app] Starting request: urn:request:d86b2162-5b8f-4d06-9878-f3b33ff49623 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:23:22,185 [206] [DEBUG] [app] Ending request: urn:request:d86b2162-5b8f-4d06-9878-f3b33ff49623 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d86b2162-5b8f-4d06-9878-f3b33ff49623', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:23:22,185 [204] [DEBUG] [app] Starting request: urn:request:7682a0cb-f4cf-4c06-af24-2abd4f4fa97a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:23:22,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:23:22,185 [204] [DEBUG] [app] Ending request: urn:request:7682a0cb-f4cf-4c06-af24-2abd4f4fa97a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7682a0cb-f4cf-4c06-af24-2abd4f4fa97a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:23:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:23:22,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:23:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:22,190 [198] [DEBUG] [app] Starting request: urn:request:65179f22-b5dd-4606-9833-741e425d7634 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:23:22,191 [198] [DEBUG] [app] Ending request: urn:request:65179f22-b5dd-4606-9833-741e425d7634 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:65179f22-b5dd-4606-9833-741e425d7634', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:23:22,191 [200] [DEBUG] [app] Starting request: urn:request:b7a200dc-f957-4f00-ab0e-80cd172ff52d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:23:22,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:23:22,192 [200] [DEBUG] [app] Ending request: urn:request:b7a200dc-f957-4f00-ab0e-80cd172ff52d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b7a200dc-f957-4f00-ab0e-80cd172ff52d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:23:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:23:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:23:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:23:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:23:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:23:22,202 [197] [DEBUG] [app] Ending request: urn:request:c97c843f-4067-4cc9-b4e0-043ad637eb17 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c97c843f-4067-4cc9-b4e0-043ad637eb17', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:23:22,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:23:22,202 [199] [DEBUG] [app] Ending request: urn:request:e43c9755-6866-45dd-a27e-317f8996a55e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e43c9755-6866-45dd-a27e-317f8996a55e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:23:22,203 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:23:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:23:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:23:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:23:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:23:22,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:23:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:23:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:23:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:28.874906+00:00 (in 6.580621 seconds) chunkcleanupworker stdout | 2023-08-08 06:23:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:32 UTC)" (scheduled at 2023-08-08 06:23:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:23:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:23:22,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 22, 294825), True, datetime.datetime(2023, 8, 8, 6, 23, 22, 294825), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:23:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:23:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:23:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:23:24,202 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:23:24,266 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:23:24,275 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:23:24,275 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:23:24,279 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:23:25,270 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:23:25,273 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:23:25,277 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:23:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:23:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:35.371682+00:00 (in 9.999501 seconds) notificationworker stdout | 2023-08-08 06:23:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:35 UTC)" (scheduled at 2023-08-08 06:23:25.371682+00:00) notificationworker stdout | 2023-08-08 06:23:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:23:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 25, 372760), True, datetime.datetime(2023, 8, 8, 6, 23, 25, 372760), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:23:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:23:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:23:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:23:25,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:33.514094+00:00 (in 7.999261 seconds) exportactionlogsworker stdout | 2023-08-08 06:23:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:25 UTC)" (scheduled at 2023-08-08 06:23:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:23:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:23:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:23:26,053 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:23:26,387 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:23:26,526 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:23:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:23:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:49.687363+00:00 (in 22.796109 seconds) securityworker stdout | 2023-08-08 06:23:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:56 UTC)" (scheduled at 2023-08-08 06:23:26.890741+00:00) securityworker stdout | 2023-08-08 06:23:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:23:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:23:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:23:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:23:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:23:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:23:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:23:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:23:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:23:26,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:23:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 18, 26, 896445), 1, 2]) securityworker stdout | 2023-08-08 06:23:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 18, 26, 896445), 1, 2]) securityworker stdout | 2023-08-08 06:23:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:23:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:23:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:23:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:23:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:23:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:23:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:23:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:23:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:23:27,520 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:23:27,666 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:23:27,839 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:23:27,908 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:23:27,930 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:23:27,939 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:23:28,092 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:23:28,268 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:23:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:23:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:32.293757+00:00 (in 3.418330 seconds) chunkcleanupworker stdout | 2023-08-08 06:23:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:28 UTC)" (scheduled at 2023-08-08 06:23:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:23:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:23:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:23:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:23:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:39.417660+00:00 (in 9.999483 seconds) gcworker stdout | 2023-08-08 06:23:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:39 UTC)" (scheduled at 2023-08-08 06:23:29.417660+00:00) gcworker stdout | 2023-08-08 06:23:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266209418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:23:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:23:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:23:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:23:29,568 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:23:29,783 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:23:30,776 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:23:31,667 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:23:31,786 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:23:32,201 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:23:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:23:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:42.293757+00:00 (in 9.999455 seconds) chunkcleanupworker stdout | 2023-08-08 06:23:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:42 UTC)" (scheduled at 2023-08-08 06:23:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:23:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:23:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 32, 294820), True, datetime.datetime(2023, 8, 8, 6, 23, 32, 294820), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:23:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:23:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:23:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:42 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:23:32,660 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:23:32,989 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:23:33,359 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} exportactionlogsworker stdout | 2023-08-08 06:23:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:23:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:47.398551+00:00 (in 13.883793 seconds) exportactionlogsworker stdout | 2023-08-08 06:23:33,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:28:33 UTC)" (scheduled at 2023-08-08 06:23:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:23:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 23, 33, 515118), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:23:33,522 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 23, 33, 515118), True, datetime.datetime(2023, 8, 8, 6, 23, 33, 515118), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:23:33,523 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 23, 33, 515118), True, datetime.datetime(2023, 8, 8, 6, 23, 33, 515118), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 23, 33, 515118), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:23:33,524 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:23:33,525 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:28:33 UTC)" executed successfully gunicorn-registry stdout | 2023-08-08 06:23:33,575 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:23:33,584 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:23:33,587 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:23:33,592 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:23:33,595 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:23:33,598 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:23:33,605 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:23:33,609 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:23:33,614 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:23:34,289 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:23:34,298 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:23:34,298 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:23:34,302 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:23:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:45.371682+00:00 (in 9.999475 seconds) notificationworker stdout | 2023-08-08 06:23:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:45 UTC)" (scheduled at 2023-08-08 06:23:35.371682+00:00) notificationworker stdout | 2023-08-08 06:23:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:23:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 35, 372752), True, datetime.datetime(2023, 8, 8, 6, 23, 35, 372752), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:23:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:23:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:23:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:23:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:54.393627+00:00 (in 17.813235 seconds) namespacegcworker stdout | 2023-08-08 06:23:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:36 UTC)" (scheduled at 2023-08-08 06:23:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:23:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:23:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:23:37,179 [199] [DEBUG] [app] Starting request: urn:request:d6138bb2-d0c5-4243-9be5-3daf2b63ea60 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:23:37,179 [197] [DEBUG] [app] Starting request: urn:request:97f0502c-14ea-47fc-990a-4257c897c7aa (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:23:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:23:37,184 [203] [DEBUG] [app] Starting request: urn:request:3c333f48-a3a8-45eb-9f07-c859030ecbe4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:23:37,185 [206] [DEBUG] [app] Starting request: urn:request:fb733a5a-7ae6-4c49-998e-4aa3a883bea4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:23:37,185 [203] [DEBUG] [app] Ending request: urn:request:3c333f48-a3a8-45eb-9f07-c859030ecbe4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3c333f48-a3a8-45eb-9f07-c859030ecbe4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:23:37,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:23:37,186 [206] [DEBUG] [app] Ending request: urn:request:fb733a5a-7ae6-4c49-998e-4aa3a883bea4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fb733a5a-7ae6-4c49-998e-4aa3a883bea4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:23:37,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:23:37,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:23:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:37,190 [200] [DEBUG] [app] Starting request: urn:request:6a0ea0ac-e76c-4576-aef9-27a5d6ccc8be (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:23:37,191 [200] [DEBUG] [app] Ending request: urn:request:6a0ea0ac-e76c-4576-aef9-27a5d6ccc8be (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6a0ea0ac-e76c-4576-aef9-27a5d6ccc8be', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:23:37,191 [198] [DEBUG] [app] Starting request: urn:request:831c1d95-1089-4bc1-9246-7fded6ac8a2c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:23:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:23:37,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:37,192 [198] [DEBUG] [app] Ending request: urn:request:831c1d95-1089-4bc1-9246-7fded6ac8a2c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:831c1d95-1089-4bc1-9246-7fded6ac8a2c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:23:37,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:23:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:23:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:23:37,196 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:23:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:23:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:23:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:07.197412+00:00 (in 29.999499 seconds) buildlogsarchiver stdout | 2023-08-08 06:23:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:07 UTC)" (scheduled at 2023-08-08 06:23:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:23:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 23, 37, 198352), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:23:37,202 [199] [DEBUG] [app] Ending request: urn:request:d6138bb2-d0c5-4243-9be5-3daf2b63ea60 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d6138bb2-d0c5-4243-9be5-3daf2b63ea60', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:23:37,202 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:23:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:23:37,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:23:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:23:37,202 [197] [DEBUG] [app] Ending request: urn:request:97f0502c-14ea-47fc-990a-4257c897c7aa (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:97f0502c-14ea-47fc-990a-4257c897c7aa', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:23:37,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:23:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:23:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:23:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:23:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:23:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:23:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:23:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:23:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:23:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:23:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:49.417660+00:00 (in 9.999435 seconds) gcworker stdout | 2023-08-08 06:23:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:49 UTC)" (scheduled at 2023-08-08 06:23:39.417660+00:00) gcworker stdout | 2023-08-08 06:23:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266219418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:23:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:23:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:23:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:23:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:23:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:52.293757+00:00 (in 9.999459 seconds) chunkcleanupworker stdout | 2023-08-08 06:23:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:52 UTC)" (scheduled at 2023-08-08 06:23:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:23:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:23:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 42, 294808), True, datetime.datetime(2023, 8, 8, 6, 23, 42, 294808), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:23:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:23:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:23:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:52 UTC)" executed successfully builder stdout | 2023-08-08 06:23:44,312 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:23:44,321 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:23:44,321 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:23:44,326 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:23:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:49.178248+00:00 (in 3.805994 seconds) notificationworker stdout | 2023-08-08 06:23:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:55 UTC)" (scheduled at 2023-08-08 06:23:45.371682+00:00) notificationworker stdout | 2023-08-08 06:23:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:23:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 45, 373263), True, datetime.datetime(2023, 8, 8, 6, 23, 45, 373263), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:23:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:23:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:23:47,399 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:23:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:25.514341+00:00 (in 38.115113 seconds) exportactionlogsworker stdout | 2023-08-08 06:23:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:47 UTC)" (scheduled at 2023-08-08 06:23:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:23:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:23:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 47, 399586), True, datetime.datetime(2023, 8, 8, 6, 23, 47, 399586), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:23:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:23:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:23:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:23:48,822 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} notificationworker stdout | 2023-08-08 06:23:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:23:49,178 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:55.371682+00:00 (in 6.192861 seconds) notificationworker stdout | 2023-08-08 06:23:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:28:49 UTC)" (scheduled at 2023-08-08 06:23:49.178248+00:00) notificationworker stdout | 2023-08-08 06:23:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 23, 49, 179455), 'notification/%']) notificationworker stdout | 2023-08-08 06:23:49,187 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 23, 49, 179455), True, datetime.datetime(2023, 8, 8, 6, 23, 49, 179455), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:23:49,188 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 23, 49, 179455), True, datetime.datetime(2023, 8, 8, 6, 23, 49, 179455), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 23, 49, 179455), 'notification/%']) notificationworker stdout | 2023-08-08 06:23:49,190 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:49,190 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:28:49 UTC)" executed successfully gcworker stdout | 2023-08-08 06:23:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:23:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:59.417660+00:00 (in 9.999451 seconds) gcworker stdout | 2023-08-08 06:23:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:59 UTC)" (scheduled at 2023-08-08 06:23:49.417660+00:00) gcworker stdout | 2023-08-08 06:23:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266229418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:23:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:23:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:23:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:23:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:23:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:23:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:23:56.890741+00:00 (in 7.202900 seconds) securityworker stdout | 2023-08-08 06:23:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:19 UTC)" (scheduled at 2023-08-08 06:23:49.687363+00:00) securityworker stdout | 2023-08-08 06:23:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:23:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:23:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:23:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:23:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:23:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:23:50,040 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:23:51,072 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:23:51,593 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:23:52,180 [198] [DEBUG] [app] Starting request: urn:request:7b5b71a9-bde6-4a9b-a710-63b5e29ed19d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:23:52,180 [200] [DEBUG] [app] Starting request: urn:request:f9b94c87-1b3d-4748-aea2-bca2f22d77df (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:23:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:52,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:23:52,185 [209] [DEBUG] [app] Starting request: urn:request:04b16e23-07a1-4910-b511-4b279fd12186 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:23:52,186 [209] [DEBUG] [app] Ending request: urn:request:04b16e23-07a1-4910-b511-4b279fd12186 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:04b16e23-07a1-4910-b511-4b279fd12186', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:23:52,186 [208] [DEBUG] [app] Starting request: urn:request:f149350b-c1e2-4c09-abf8-92143c1209bb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:23:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:23:52,186 [208] [DEBUG] [app] Ending request: urn:request:f149350b-c1e2-4c09-abf8-92143c1209bb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f149350b-c1e2-4c09-abf8-92143c1209bb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:23:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:23:52,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:23:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:52,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:52,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:23:52,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:52,191 [200] [DEBUG] [app] Starting request: urn:request:66d5f27b-a8af-4d9d-9b13-1bae74365e01 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:23:52,191 [200] [DEBUG] [app] Ending request: urn:request:66d5f27b-a8af-4d9d-9b13-1bae74365e01 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:66d5f27b-a8af-4d9d-9b13-1bae74365e01', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:23:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:23:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:52,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:23:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:23:52,193 [199] [DEBUG] [app] Starting request: urn:request:0ea54c5d-f1ba-4fef-bd1f-08d87823043d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:23:52,194 [199] [DEBUG] [app] Ending request: urn:request:0ea54c5d-f1ba-4fef-bd1f-08d87823043d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0ea54c5d-f1ba-4fef-bd1f-08d87823043d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:23:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:23:52,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:23:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:23:52,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:23:52,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:23:52,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:23:52,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:23:52,202 [198] [DEBUG] [app] Ending request: urn:request:7b5b71a9-bde6-4a9b-a710-63b5e29ed19d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7b5b71a9-bde6-4a9b-a710-63b5e29ed19d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:23:52,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:23:52,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:23:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:23:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:23:52,204 [200] [DEBUG] [app] Ending request: urn:request:f9b94c87-1b3d-4748-aea2-bca2f22d77df (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f9b94c87-1b3d-4748-aea2-bca2f22d77df', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:23:52,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:23:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:23:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:23:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:23:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:23:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:02.293757+00:00 (in 9.999454 seconds) chunkcleanupworker stdout | 2023-08-08 06:23:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:02 UTC)" (scheduled at 2023-08-08 06:23:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:23:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:23:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 52, 294882), True, datetime.datetime(2023, 8, 8, 6, 23, 52, 294882), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:23:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:23:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:23:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:23:54,212 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:23:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:23:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:13.097375+00:00 (in 18.812726 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:23:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:54 UTC)" (scheduled at 2023-08-08 06:23:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:23:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:23:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:54 UTC)" executed successfully builder stdout | 2023-08-08 06:23:54,335 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:23:54,344 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:23:54,344 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:23:54,349 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:23:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:23:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:36.579841+00:00 (in 42.185530 seconds) namespacegcworker stdout | 2023-08-08 06:23:54,395 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:54 UTC)" (scheduled at 2023-08-08 06:23:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:23:54,395 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:23:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 54, 395385), True, datetime.datetime(2023, 8, 8, 6, 23, 54, 395385), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:23:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:23:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:23:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:24:54 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:23:55,281 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:23:55,287 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:23:55,290 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:23:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:23:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:02.178530+00:00 (in 6.806335 seconds) notificationworker stdout | 2023-08-08 06:23:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:05 UTC)" (scheduled at 2023-08-08 06:23:55.371682+00:00) notificationworker stdout | 2023-08-08 06:23:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:23:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 23, 55, 372800), True, datetime.datetime(2023, 8, 8, 6, 23, 55, 372800), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:23:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:23:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:23:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:23:56,091 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:23:56,425 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:23:56,555 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:23:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:23:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:19.687363+00:00 (in 22.796089 seconds) securityworker stdout | 2023-08-08 06:23:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:26 UTC)" (scheduled at 2023-08-08 06:23:56.890741+00:00) securityworker stdout | 2023-08-08 06:23:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:23:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:23:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:23:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:23:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:23:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:23:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:23:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:23:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:23:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:23:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 18, 56, 896634), 1, 2]) securityworker stdout | 2023-08-08 06:23:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 18, 56, 896634), 1, 2]) securityworker stdout | 2023-08-08 06:23:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:23:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:23:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:23:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:23:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:23:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:23:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:23:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:23:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:23:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:23:57,557 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:23:57,674 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:23:57,852 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:23:57,923 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:23:57,944 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:23:57,953 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:23:58,100 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:23:58,305 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:23:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:23:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:09.417660+00:00 (in 9.999458 seconds) gcworker stdout | 2023-08-08 06:23:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:09 UTC)" (scheduled at 2023-08-08 06:23:59.417660+00:00) gcworker stdout | 2023-08-08 06:23:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266239418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:23:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:23:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:23:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:23:59,592 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:23:59,800 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:24:00,800 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:24:01,689 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:24:01,824 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} notificationworker stdout | 2023-08-08 06:24:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:24:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:05.371682+00:00 (in 3.192577 seconds) notificationworker stdout | 2023-08-08 06:24:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:02 UTC)" (scheduled at 2023-08-08 06:24:02.178530+00:00) notificationworker stdout | 2023-08-08 06:24:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:24:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:02 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:24:02,240 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:24:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:24:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:12.293757+00:00 (in 9.999456 seconds) chunkcleanupworker stdout | 2023-08-08 06:24:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:12 UTC)" (scheduled at 2023-08-08 06:24:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:24:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:24:02,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 2, 294998), True, datetime.datetime(2023, 8, 8, 6, 24, 2, 294998), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:24:02,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:24:02,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:24:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:12 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:24:02,689 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:24:03,017 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:24:03,397 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:24:03,587 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:24:03,596 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:24:03,599 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:24:03,603 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:24:03,607 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:24:03,610 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:24:03,615 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:24:03,618 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:24:03,626 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:24:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:24:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:09.793658+00:00 (in 5.605702 seconds) repositorygcworker stdout | 2023-08-08 06:24:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:04 UTC)" (scheduled at 2023-08-08 06:24:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:24:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:24:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:04 UTC)" executed successfully builder stdout | 2023-08-08 06:24:04,359 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:24:04,368 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:24:04,368 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:24:04,372 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:24:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:15.371682+00:00 (in 9.999420 seconds) notificationworker stdout | 2023-08-08 06:24:05,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:15 UTC)" (scheduled at 2023-08-08 06:24:05.371682+00:00) notificationworker stdout | 2023-08-08 06:24:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:24:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 5, 373317), True, datetime.datetime(2023, 8, 8, 6, 24, 5, 373317), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:24:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:24:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:24:07,179 [199] [DEBUG] [app] Starting request: urn:request:9f6c5d2f-4d7c-43f7-b5a0-5acf1fbc23b6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:24:07,179 [197] [DEBUG] [app] Starting request: urn:request:af7231d9-8fae-40ce-8b4b-705250bd8945 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:24:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:24:07,185 [203] [DEBUG] [app] Starting request: urn:request:5818cb77-6e0c-43cc-bf93-aa42943af598 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:24:07,185 [203] [DEBUG] [app] Ending request: urn:request:5818cb77-6e0c-43cc-bf93-aa42943af598 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5818cb77-6e0c-43cc-bf93-aa42943af598', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:24:07,185 [210] [DEBUG] [app] Starting request: urn:request:15ba1c3b-a310-485f-ad9c-b93c930e0154 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:24:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:24:07,186 [210] [DEBUG] [app] Ending request: urn:request:15ba1c3b-a310-485f-ad9c-b93c930e0154 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:15ba1c3b-a310-485f-ad9c-b93c930e0154', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:24:07,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:24:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:07,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:07,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:07,192 [199] [DEBUG] [app] Starting request: urn:request:29a8383d-6338-4e81-92f5-16d497b75f65 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:24:07,192 [200] [DEBUG] [app] Starting request: urn:request:cc7d82cc-a398-4d6a-a735-c2fcf50715a4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:24:07,192 [199] [DEBUG] [app] Ending request: urn:request:29a8383d-6338-4e81-92f5-16d497b75f65 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:29a8383d-6338-4e81-92f5-16d497b75f65', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:24:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:07,192 [200] [DEBUG] [app] Ending request: urn:request:cc7d82cc-a398-4d6a-a735-c2fcf50715a4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:cc7d82cc-a398-4d6a-a735-c2fcf50715a4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:24:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:24:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:07,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:24:07,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:24:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:24:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:37.197412+00:00 (in 29.999564 seconds) gunicorn-web stdout | 2023-08-08 06:24:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:24:07,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:24:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:37 UTC)" (scheduled at 2023-08-08 06:24:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:24:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 24, 7, 198791), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:24:07,203 [197] [DEBUG] [app] Ending request: urn:request:af7231d9-8fae-40ce-8b4b-705250bd8945 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:af7231d9-8fae-40ce-8b4b-705250bd8945', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:24:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:24:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:24:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:24:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:24:07,204 [199] [DEBUG] [app] Ending request: urn:request:9f6c5d2f-4d7c-43f7-b5a0-5acf1fbc23b6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9f6c5d2f-4d7c-43f7-b5a0-5acf1fbc23b6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:24:07,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:24:07,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:24:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:24:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:24:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:24:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:24:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:24:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:24:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:24:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:24:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:19.417660+00:00 (in 9.999448 seconds) gcworker stdout | 2023-08-08 06:24:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:19 UTC)" (scheduled at 2023-08-08 06:24:09.417660+00:00) gcworker stdout | 2023-08-08 06:24:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266249418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:24:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:24:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:24:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:24:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:24:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:19.187230+00:00 (in 9.393020 seconds) repositorygcworker stdout | 2023-08-08 06:24:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:09 UTC)" (scheduled at 2023-08-08 06:24:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:24:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:24:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 9, 795632), True, datetime.datetime(2023, 8, 8, 6, 24, 9, 795632), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:24:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:24:09,804 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:24:09,804 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:24:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:24:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:22.293757+00:00 (in 9.999407 seconds) chunkcleanupworker stdout | 2023-08-08 06:24:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:22 UTC)" (scheduled at 2023-08-08 06:24:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:24:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:24:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 12, 294940), True, datetime.datetime(2023, 8, 8, 6, 24, 12, 294940), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:24:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:24:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:24:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:24:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:24:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:54.284093+00:00 (in 41.186084 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:24:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:13 UTC)" (scheduled at 2023-08-08 06:24:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:24:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:24:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 13, 98685), True, datetime.datetime(2023, 8, 8, 6, 24, 13, 98685), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:24:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:24:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:24:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:13 UTC)" executed successfully builder stdout | 2023-08-08 06:24:14,382 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:24:14,392 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:24:14,392 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:24:14,396 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:24:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:25.371682+00:00 (in 9.999452 seconds) notificationworker stdout | 2023-08-08 06:24:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:25 UTC)" (scheduled at 2023-08-08 06:24:15.371682+00:00) notificationworker stdout | 2023-08-08 06:24:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:24:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 15, 372692), True, datetime.datetime(2023, 8, 8, 6, 24, 15, 372692), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:24:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:24:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:24:15,832 [197] [DEBUG] [app] Starting request: urn:request:459d0764-a8c6-44e9-a068-5316aa8555da (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:24:15,832 [200] [DEBUG] [app] Starting request: urn:request:8b4adaf7-7aa9-484d-953e-6194621ffb5c (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:24:15,832 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:24:15,832 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:24:15,832 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:24:15,832 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:24:15,833 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:24:15,833 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:24:15,840 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:24:15,840 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:24:15,841 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:24:15,841 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:24:15,841 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:24:15,841 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:24:15,841 [197] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:24:15,841 [200] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:24:15,841 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:24:15,841 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:24:15,841 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:24:15,841 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:24:15,841 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:24:15,841 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:24:15,841 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:24:15,841 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:24:15,842 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 36, 15, 841598), datetime.datetime(2023, 8, 8, 6, 24, 15, 841668)]) gunicorn-web stdout | 2023-08-08 06:24:15,842 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:24:15,843 [200] [DEBUG] [app] Ending request: urn:request:8b4adaf7-7aa9-484d-953e-6194621ffb5c (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:8b4adaf7-7aa9-484d-953e-6194621ffb5c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:24:15,844 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:24:15 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 62 0.014) gunicorn-web stdout | 2023-08-08 06:24:15,844 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:24:15 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:24:15,847 [197] [DEBUG] [app] Ending request: urn:request:459d0764-a8c6-44e9-a068-5316aa8555da (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:459d0764-a8c6-44e9-a068-5316aa8555da', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:24:15,848 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:24:15 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 830 0.017) gunicorn-web stdout | 2023-08-08 06:24:15,848 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:24:15 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:24:18,844 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} repositorygcworker stdout | 2023-08-08 06:24:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:24:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:04.187429+00:00 (in 44.999654 seconds) repositorygcworker stdout | 2023-08-08 06:24:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:29:19 UTC)" (scheduled at 2023-08-08 06:24:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:24:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 24, 19, 188277), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:24:19,196 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 24, 19, 188277), True, datetime.datetime(2023, 8, 8, 6, 24, 19, 188277), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:24:19,198 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 24, 19, 188277), True, datetime.datetime(2023, 8, 8, 6, 24, 19, 188277), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 24, 19, 188277), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:24:19,199 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:24:19,200 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:29:19 UTC)" executed successfully gcworker stdout | 2023-08-08 06:24:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:24:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:29.417660+00:00 (in 9.999448 seconds) gcworker stdout | 2023-08-08 06:24:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:29 UTC)" (scheduled at 2023-08-08 06:24:19.417660+00:00) gcworker stdout | 2023-08-08 06:24:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266259418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:24:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:24:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:24:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:24:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:24:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:26.890741+00:00 (in 7.202764 seconds) securityworker stdout | 2023-08-08 06:24:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:49 UTC)" (scheduled at 2023-08-08 06:24:19.687363+00:00) securityworker stdout | 2023-08-08 06:24:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:24:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:24:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:24:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:24:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:24:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:24:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 19, 19, 692069), 1, 2]) securityworker stdout | 2023-08-08 06:24:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 19, 19, 692069), 1, 2]) securityworker stdout | 2023-08-08 06:24:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:24:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:24:20,077 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:24:21,108 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:24:21,609 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:24:22,179 [197] [DEBUG] [app] Starting request: urn:request:61fdace2-612d-4b6b-be6c-a9f2f01bcfce (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:24:22,179 [199] [DEBUG] [app] Starting request: urn:request:98bcb03a-919c-487b-94cc-74d1c65a0ac5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:24:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:24:22,184 [210] [DEBUG] [app] Starting request: urn:request:bc4146d7-419d-4507-b6dc-bdf3acd3f5ff (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:24:22,184 [208] [DEBUG] [app] Starting request: urn:request:7f66c1bd-34fa-49ea-9bff-e4a337619e01 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:24:22,185 [210] [DEBUG] [app] Ending request: urn:request:bc4146d7-419d-4507-b6dc-bdf3acd3f5ff (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:bc4146d7-419d-4507-b6dc-bdf3acd3f5ff', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:24:22,185 [208] [DEBUG] [app] Ending request: urn:request:7f66c1bd-34fa-49ea-9bff-e4a337619e01 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7f66c1bd-34fa-49ea-9bff-e4a337619e01', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:24:22,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:24:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:24:22,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:22,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:22,190 [197] [DEBUG] [app] Starting request: urn:request:23638914-ea44-4d86-95f4-da86efba6090 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:24:22,191 [197] [DEBUG] [app] Ending request: urn:request:23638914-ea44-4d86-95f4-da86efba6090 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:23638914-ea44-4d86-95f4-da86efba6090', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:24:22,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:22,191 [200] [DEBUG] [app] Starting request: urn:request:9f9d53b2-aa52-452d-a282-4762c39432b6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.000) gunicorn-web stdout | 2023-08-08 06:24:22,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:22,191 [200] [DEBUG] [app] Ending request: urn:request:9f9d53b2-aa52-452d-a282-4762c39432b6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9f9d53b2-aa52-452d-a282-4762c39432b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:24:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:24:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:24:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:24:22,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:24:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:24:22,201 [197] [DEBUG] [app] Ending request: urn:request:61fdace2-612d-4b6b-be6c-a9f2f01bcfce (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:61fdace2-612d-4b6b-be6c-a9f2f01bcfce', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:24:22,201 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:24:22,202 [199] [DEBUG] [app] Ending request: urn:request:98bcb03a-919c-487b-94cc-74d1c65a0ac5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:98bcb03a-919c-487b-94cc-74d1c65a0ac5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:24:22,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:24:22,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:24:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:24:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:24:22,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:24:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:24:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) chunkcleanupworker stdout | 2023-08-08 06:24:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:24:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:28.874906+00:00 (in 6.580538 seconds) chunkcleanupworker stdout | 2023-08-08 06:24:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:32 UTC)" (scheduled at 2023-08-08 06:24:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:24:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:24:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 22, 294872), True, datetime.datetime(2023, 8, 8, 6, 24, 22, 294872), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:24:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:24:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:24:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:24:24,226 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:24:24,406 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:24:24,416 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:24:24,416 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:24:24,421 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:24:25,290 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:24:25,296 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:24:25,299 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:24:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:24:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:35.371682+00:00 (in 9.999435 seconds) notificationworker stdout | 2023-08-08 06:24:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:35 UTC)" (scheduled at 2023-08-08 06:24:25.371682+00:00) notificationworker stdout | 2023-08-08 06:24:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:24:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 25, 372745), True, datetime.datetime(2023, 8, 8, 6, 24, 25, 372745), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:24:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:24:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:24:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:24:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:47.398551+00:00 (in 21.883674 seconds) exportactionlogsworker stdout | 2023-08-08 06:24:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:25 UTC)" (scheduled at 2023-08-08 06:24:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:24:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:24:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:24:26,107 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:24:26,462 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:24:26,593 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:24:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:24:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:49.687363+00:00 (in 22.796094 seconds) securityworker stdout | 2023-08-08 06:24:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:56 UTC)" (scheduled at 2023-08-08 06:24:26.890741+00:00) securityworker stdout | 2023-08-08 06:24:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:24:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:24:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:24:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:24:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:24:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:24:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:24:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:24:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:24:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:24:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 19, 26, 896593), 1, 2]) securityworker stdout | 2023-08-08 06:24:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 19, 26, 896593), 1, 2]) securityworker stdout | 2023-08-08 06:24:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:24:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:24:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:24:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:24:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:24:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:24:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:24:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:24:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:24:27,586 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:24:27,683 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:24:27,865 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:24:27,937 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:24:27,957 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:24:27,966 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:24:28,116 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:24:28,314 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:24:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:24:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:32.293757+00:00 (in 3.418285 seconds) chunkcleanupworker stdout | 2023-08-08 06:24:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:28 UTC)" (scheduled at 2023-08-08 06:24:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:24:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:24:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:24:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:24:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:39.417660+00:00 (in 9.999414 seconds) gcworker stdout | 2023-08-08 06:24:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:39 UTC)" (scheduled at 2023-08-08 06:24:29.417660+00:00) gcworker stdout | 2023-08-08 06:24:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266269418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:24:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:24:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:24:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:24:29,622 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:24:29,816 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:24:30,825 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:24:31,699 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:24:31,856 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:24:32,250 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:24:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:24:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:42.293757+00:00 (in 9.999430 seconds) chunkcleanupworker stdout | 2023-08-08 06:24:32,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:42 UTC)" (scheduled at 2023-08-08 06:24:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:24:32,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:24:32,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 32, 295435), True, datetime.datetime(2023, 8, 8, 6, 24, 32, 295435), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:24:32,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:24:32,305 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:24:32,305 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:42 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:24:32,697 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:24:33,055 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:24:33,435 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:24:33,600 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:24:33,608 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:24:33,610 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:24:33,613 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:24:33,618 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:24:33,621 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:24:33,624 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:24:33,629 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:24:33,635 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:24:34,431 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:24:34,440 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:24:34,440 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:24:34,445 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:24:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:45.371682+00:00 (in 9.999466 seconds) notificationworker stdout | 2023-08-08 06:24:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:45 UTC)" (scheduled at 2023-08-08 06:24:35.371682+00:00) notificationworker stdout | 2023-08-08 06:24:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:24:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 35, 372817), True, datetime.datetime(2023, 8, 8, 6, 24, 35, 372817), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:24:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:24:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:24:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:24:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:39.579576+00:00 (in 2.999169 seconds) namespacegcworker stdout | 2023-08-08 06:24:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:36 UTC)" (scheduled at 2023-08-08 06:24:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:24:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:24:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:24:37,179 [200] [DEBUG] [app] Starting request: urn:request:e3d8734b-7f95-4c7a-95c1-13fec65f3939 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:24:37,180 [199] [DEBUG] [app] Starting request: urn:request:ee8474ba-1b56-4bd2-9356-b37f828cae34 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:24:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:37,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:37,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:24:37,185 [209] [DEBUG] [app] Starting request: urn:request:5e4f982c-7764-4b2a-9929-f82e788c9feb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:24:37,186 [209] [DEBUG] [app] Ending request: urn:request:5e4f982c-7764-4b2a-9929-f82e788c9feb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5e4f982c-7764-4b2a-9929-f82e788c9feb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:24:37,186 [203] [DEBUG] [app] Starting request: urn:request:8ad02d29-79fc-419a-843d-28238f3d4ea3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:24:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:24:37,186 [203] [DEBUG] [app] Ending request: urn:request:8ad02d29-79fc-419a-843d-28238f3d4ea3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8ad02d29-79fc-419a-843d-28238f3d4ea3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:24:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:24:37,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:37,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:37,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:37,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:37,191 [200] [DEBUG] [app] Starting request: urn:request:b6086600-ccc2-4088-acd8-43d24555c4cf (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:24:37,191 [197] [DEBUG] [app] Starting request: urn:request:880613fc-4afd-4a1c-a139-2daf17557e76 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:24:37,192 [200] [DEBUG] [app] Ending request: urn:request:b6086600-ccc2-4088-acd8-43d24555c4cf (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b6086600-ccc2-4088-acd8-43d24555c4cf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:24:37,192 [197] [DEBUG] [app] Ending request: urn:request:880613fc-4afd-4a1c-a139-2daf17557e76 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:880613fc-4afd-4a1c-a139-2daf17557e76', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:24:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:37,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:37,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:24:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:24:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:24:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:24:37,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:24:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:07.197412+00:00 (in 29.999544 seconds) buildlogsarchiver stdout | 2023-08-08 06:24:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:07 UTC)" (scheduled at 2023-08-08 06:24:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:24:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 24, 37, 198370), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:24:37,203 [199] [DEBUG] [app] Ending request: urn:request:ee8474ba-1b56-4bd2-9356-b37f828cae34 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ee8474ba-1b56-4bd2-9356-b37f828cae34', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:24:37,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:24:37,203 [200] [DEBUG] [app] Ending request: urn:request:e3d8734b-7f95-4c7a-95c1-13fec65f3939 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e3d8734b-7f95-4c7a-95c1-13fec65f3939', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:24:37,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:24:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:24:37,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:24:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:24:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:24:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:24:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:24:37,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:24:37,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:24:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:24:37,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:24:37,209 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:24:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:24:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:49.417660+00:00 (in 9.999449 seconds) gcworker stdout | 2023-08-08 06:24:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:49 UTC)" (scheduled at 2023-08-08 06:24:39.417660+00:00) gcworker stdout | 2023-08-08 06:24:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266279419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:24:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:24:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:24:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:24:39,579 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:24:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:54.393627+00:00 (in 14.813506 seconds) namespacegcworker stdout | 2023-08-08 06:24:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:29:39 UTC)" (scheduled at 2023-08-08 06:24:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:24:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 24, 39, 580886), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:24:39,587 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 24, 39, 580886), True, datetime.datetime(2023, 8, 8, 6, 24, 39, 580886), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:24:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 24, 39, 580886), True, datetime.datetime(2023, 8, 8, 6, 24, 39, 580886), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 24, 39, 580886), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:24:39,590 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:24:39,590 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:29:39 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:24:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:24:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:46.874674+00:00 (in 4.580370 seconds) chunkcleanupworker stdout | 2023-08-08 06:24:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:52 UTC)" (scheduled at 2023-08-08 06:24:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:24:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:24:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 42, 294814), True, datetime.datetime(2023, 8, 8, 6, 24, 42, 294814), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:24:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:24:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:24:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:52 UTC)" executed successfully builder stdout | 2023-08-08 06:24:44,455 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:24:44,464 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:24:44,464 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:24:44,468 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:24:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:55.371682+00:00 (in 9.999440 seconds) notificationworker stdout | 2023-08-08 06:24:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:55 UTC)" (scheduled at 2023-08-08 06:24:45.371682+00:00) notificationworker stdout | 2023-08-08 06:24:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:24:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 45, 372729), True, datetime.datetime(2023, 8, 8, 6, 24, 45, 372729), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:24:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:24:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:24:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:24:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:52.293757+00:00 (in 5.418555 seconds) chunkcleanupworker stdout | 2023-08-08 06:24:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:29:46 UTC)" (scheduled at 2023-08-08 06:24:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:24:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 24, 46, 875756), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:24:46,883 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 24, 46, 875756), True, datetime.datetime(2023, 8, 8, 6, 24, 46, 875756), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:24:46,885 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 24, 46, 875756), True, datetime.datetime(2023, 8, 8, 6, 24, 46, 875756), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 24, 46, 875756), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:24:46,886 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:24:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:29:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:24:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:24:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:25.514341+00:00 (in 38.115223 seconds) exportactionlogsworker stdout | 2023-08-08 06:24:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:47 UTC)" (scheduled at 2023-08-08 06:24:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:24:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:24:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 47, 399506), True, datetime.datetime(2023, 8, 8, 6, 24, 47, 399506), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:24:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:24:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:24:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:24:48,878 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:24:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:24:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:59.417660+00:00 (in 9.999471 seconds) gcworker stdout | 2023-08-08 06:24:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:59 UTC)" (scheduled at 2023-08-08 06:24:49.417660+00:00) gcworker stdout | 2023-08-08 06:24:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266289418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:24:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:24:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:24:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:24:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:24:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:24:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:24:56.890741+00:00 (in 7.202867 seconds) securityworker stdout | 2023-08-08 06:24:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:19 UTC)" (scheduled at 2023-08-08 06:24:49.687363+00:00) securityworker stdout | 2023-08-08 06:24:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:24:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:24:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:24:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:24:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:24:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:24:50,100 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:24:51,144 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:24:51,645 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:24:52,180 [200] [DEBUG] [app] Starting request: urn:request:4ef69c1b-6af4-4f02-a471-23f2cdbc22c7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:24:52,180 [197] [DEBUG] [app] Starting request: urn:request:ecb41171-8177-4dd9-a73d-ce215d34720a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:24:52,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:52,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:52,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:24:52,185 [203] [DEBUG] [app] Starting request: urn:request:88b44712-5c26-4639-b5c9-68a70b0d88a6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:24:52,186 [203] [DEBUG] [app] Ending request: urn:request:88b44712-5c26-4639-b5c9-68a70b0d88a6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:88b44712-5c26-4639-b5c9-68a70b0d88a6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:24:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:24:52,186 [205] [DEBUG] [app] Starting request: urn:request:44746ad1-8e00-49c1-8367-648ae1f04cc3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:24:52,187 [205] [DEBUG] [app] Ending request: urn:request:44746ad1-8e00-49c1-8367-648ae1f04cc3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:44746ad1-8e00-49c1-8367-648ae1f04cc3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:24:52,187 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:52,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:24:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:52,191 [197] [DEBUG] [app] Starting request: urn:request:bd56006a-f2cf-4225-a279-58e750863150 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:24:52,191 [197] [DEBUG] [app] Ending request: urn:request:bd56006a-f2cf-4225-a279-58e750863150 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bd56006a-f2cf-4225-a279-58e750863150', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:24:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:24:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:24:52,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:24:52,193 [197] [DEBUG] [app] Starting request: urn:request:c894176f-b255-4a65-8afa-f8b5addb75ab (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:24:52,193 [197] [DEBUG] [app] Ending request: urn:request:c894176f-b255-4a65-8afa-f8b5addb75ab (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c894176f-b255-4a65-8afa-f8b5addb75ab', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:24:52,194 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:24:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:24:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:24:52,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:24:52,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:24:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:24:52,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:24:52,203 [200] [DEBUG] [app] Ending request: urn:request:4ef69c1b-6af4-4f02-a471-23f2cdbc22c7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4ef69c1b-6af4-4f02-a471-23f2cdbc22c7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:24:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:24:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:24:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:24:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:24:52,204 [197] [DEBUG] [app] Ending request: urn:request:ecb41171-8177-4dd9-a73d-ce215d34720a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ecb41171-8177-4dd9-a73d-ce215d34720a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:24:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:24:52,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:24:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:24:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:24:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:24:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:02.293757+00:00 (in 9.999494 seconds) chunkcleanupworker stdout | 2023-08-08 06:24:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:02 UTC)" (scheduled at 2023-08-08 06:24:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:24:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:24:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 52, 294831), True, datetime.datetime(2023, 8, 8, 6, 24, 52, 294831), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:24:52,301 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:24:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:24:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:02 UTC)" executed successfully gcworker stdout | 2023-08-08 06:24:54,264 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2023-08-08 06:24:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:24:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:13.097375+00:00 (in 18.812570 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:24:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:54 UTC)" (scheduled at 2023-08-08 06:24:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:24:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:24:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:24:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:24:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:36.579841+00:00 (in 42.185555 seconds) namespacegcworker stdout | 2023-08-08 06:24:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:54 UTC)" (scheduled at 2023-08-08 06:24:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:24:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:24:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 54, 394722), True, datetime.datetime(2023, 8, 8, 6, 24, 54, 394722), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:24:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:24:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:24:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:25:54 UTC)" executed successfully builder stdout | 2023-08-08 06:24:54,478 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:24:54,489 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:24:54,489 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:24:54,495 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:24:55,299 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:24:55,303 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:24:55,306 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:24:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:24:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:02.178530+00:00 (in 6.806310 seconds) notificationworker stdout | 2023-08-08 06:24:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:05 UTC)" (scheduled at 2023-08-08 06:24:55.371682+00:00) notificationworker stdout | 2023-08-08 06:24:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:24:55,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 24, 55, 373204), True, datetime.datetime(2023, 8, 8, 6, 24, 55, 373204), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:24:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:24:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:24:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:24:56,121 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:24:56,500 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:24:56,618 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:24:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:24:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:19.687363+00:00 (in 22.796087 seconds) securityworker stdout | 2023-08-08 06:24:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:26 UTC)" (scheduled at 2023-08-08 06:24:56.890741+00:00) securityworker stdout | 2023-08-08 06:24:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:24:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:24:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:24:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:24:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:24:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:24:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:24:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:24:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:24:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:24:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 19, 56, 897028), 1, 2]) securityworker stdout | 2023-08-08 06:24:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 19, 56, 897028), 1, 2]) securityworker stdout | 2023-08-08 06:24:56,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:24:56,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:24:56,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:24:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:24:56,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:24:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:24:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:24:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:24:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:24:56,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:24:57,611 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:24:57,691 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:24:57,878 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:24:57,952 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:24:57,971 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:24:57,981 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:24:58,131 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:24:58,327 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:24:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:24:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:09.417660+00:00 (in 9.999443 seconds) gcworker stdout | 2023-08-08 06:24:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:09 UTC)" (scheduled at 2023-08-08 06:24:59.417660+00:00) gcworker stdout | 2023-08-08 06:24:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266299418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:24:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:24:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:24:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:24:59,660 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:24:59,838 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:25:00,834 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:25:01,735 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:25:01,890 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} notificationworker stdout | 2023-08-08 06:25:02,179 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:25:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:05.371682+00:00 (in 3.191981 seconds) notificationworker stdout | 2023-08-08 06:25:02,180 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:02 UTC)" (scheduled at 2023-08-08 06:25:02.178530+00:00) notificationworker stdout | 2023-08-08 06:25:02,180 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:25:02,180 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:02 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:25:02,288 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} chunkcleanupworker stdout | 2023-08-08 06:25:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:25:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:12.293757+00:00 (in 9.999488 seconds) chunkcleanupworker stdout | 2023-08-08 06:25:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:12 UTC)" (scheduled at 2023-08-08 06:25:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:25:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:25:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 2, 294832), True, datetime.datetime(2023, 8, 8, 6, 25, 2, 294832), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:25:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:25:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:25:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:12 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:25:02,736 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:25:03,070 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:25:03,470 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:25:03,611 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:25:03,617 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:25:03,620 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:25:03,623 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:25:03,631 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:25:03,634 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:25:03,637 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:25:03,640 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:25:03,645 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:25:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:25:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:09.793658+00:00 (in 5.605685 seconds) repositorygcworker stdout | 2023-08-08 06:25:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:04 UTC)" (scheduled at 2023-08-08 06:25:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:25:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:25:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:04 UTC)" executed successfully builder stdout | 2023-08-08 06:25:04,505 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:25:04,515 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:25:04,515 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:25:04,520 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:25:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:15.371682+00:00 (in 9.999429 seconds) notificationworker stdout | 2023-08-08 06:25:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:15 UTC)" (scheduled at 2023-08-08 06:25:05.371682+00:00) notificationworker stdout | 2023-08-08 06:25:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:25:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 5, 372763), True, datetime.datetime(2023, 8, 8, 6, 25, 5, 372763), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:25:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:25:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:25:07,178 [197] [DEBUG] [app] Starting request: urn:request:b8bd444d-b24a-47e7-8d78-327ca87e6a96 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:25:07,179 [200] [DEBUG] [app] Starting request: urn:request:c527e8a2-8717-4890-b9eb-49eba9c57e6c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:25:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:07,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:25:07,184 [210] [DEBUG] [app] Starting request: urn:request:130c5e77-cf50-46c4-8cae-59ba96c99b50 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:25:07,185 [210] [DEBUG] [app] Ending request: urn:request:130c5e77-cf50-46c4-8cae-59ba96c99b50 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:130c5e77-cf50-46c4-8cae-59ba96c99b50', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:25:07,185 [209] [DEBUG] [app] Starting request: urn:request:22d1d79e-4820-4ed7-9fc0-504afde9c0d5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:25:07,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:25:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:25:07,185 [209] [DEBUG] [app] Ending request: urn:request:22d1d79e-4820-4ed7-9fc0-504afde9c0d5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:22d1d79e-4820-4ed7-9fc0-504afde9c0d5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:25:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:25:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:07,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:07,190 [200] [DEBUG] [app] Starting request: urn:request:e80d843c-3e56-45e6-ac40-609b42b213ea (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:25:07,191 [200] [DEBUG] [app] Ending request: urn:request:e80d843c-3e56-45e6-ac40-609b42b213ea (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e80d843c-3e56-45e6-ac40-609b42b213ea', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:07,191 [199] [DEBUG] [app] Starting request: urn:request:33019d6f-e52f-4c68-8b08-bc85141726da (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:25:07,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:25:07,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:07,191 [199] [DEBUG] [app] Ending request: urn:request:33019d6f-e52f-4c68-8b08-bc85141726da (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:33019d6f-e52f-4c68-8b08-bc85141726da', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:25:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:25:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:25:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:25:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:25:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:25:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:37.197412+00:00 (in 29.999500 seconds) buildlogsarchiver stdout | 2023-08-08 06:25:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:37 UTC)" (scheduled at 2023-08-08 06:25:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:25:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 25, 7, 198315), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:25:07,202 [197] [DEBUG] [app] Ending request: urn:request:b8bd444d-b24a-47e7-8d78-327ca87e6a96 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b8bd444d-b24a-47e7-8d78-327ca87e6a96', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:25:07,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:25:07,202 [200] [DEBUG] [app] Ending request: urn:request:c527e8a2-8717-4890-b9eb-49eba9c57e6c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c527e8a2-8717-4890-b9eb-49eba9c57e6c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:25:07,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:25:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:25:07,202 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:25:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:25:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:25:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:25:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:25:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:25:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:25:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:25:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:25:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:25:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:25:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:19.417660+00:00 (in 9.999431 seconds) gcworker stdout | 2023-08-08 06:25:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:19 UTC)" (scheduled at 2023-08-08 06:25:09.417660+00:00) gcworker stdout | 2023-08-08 06:25:09,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266309418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:25:09,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:25:09,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:25:09,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:25:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:25:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:04.187429+00:00 (in 54.393145 seconds) repositorygcworker stdout | 2023-08-08 06:25:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:09 UTC)" (scheduled at 2023-08-08 06:25:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:25:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:25:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 9, 794857), True, datetime.datetime(2023, 8, 8, 6, 25, 9, 794857), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:25:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:25:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:25:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:25:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:25:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:22.293757+00:00 (in 9.999442 seconds) chunkcleanupworker stdout | 2023-08-08 06:25:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:22 UTC)" (scheduled at 2023-08-08 06:25:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:25:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:25:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 12, 294817), True, datetime.datetime(2023, 8, 8, 6, 25, 12, 294817), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:25:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:25:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:25:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:25:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:25:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:54.284093+00:00 (in 41.186133 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:25:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:13 UTC)" (scheduled at 2023-08-08 06:25:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:25:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:25:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 13, 98728), True, datetime.datetime(2023, 8, 8, 6, 25, 13, 98728), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:25:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:25:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:25:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:13 UTC)" executed successfully builder stdout | 2023-08-08 06:25:14,530 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:25:14,539 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:25:14,540 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:25:14,544 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:25:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:25.371682+00:00 (in 9.999490 seconds) notificationworker stdout | 2023-08-08 06:25:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:25 UTC)" (scheduled at 2023-08-08 06:25:15.371682+00:00) notificationworker stdout | 2023-08-08 06:25:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:25:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 15, 372685), True, datetime.datetime(2023, 8, 8, 6, 25, 15, 372685), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:25:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:25:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:25:18,916 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:25:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:25:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:29.417660+00:00 (in 9.999318 seconds) gcworker stdout | 2023-08-08 06:25:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:29 UTC)" (scheduled at 2023-08-08 06:25:19.417660+00:00) gcworker stdout | 2023-08-08 06:25:19,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266319418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:25:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:25:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:25:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:25:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:25:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:26.890741+00:00 (in 7.202808 seconds) securityworker stdout | 2023-08-08 06:25:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:49 UTC)" (scheduled at 2023-08-08 06:25:19.687363+00:00) securityworker stdout | 2023-08-08 06:25:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:25:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:25:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:25:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:25:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:25:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:25:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 20, 19, 692494), 1, 2]) securityworker stdout | 2023-08-08 06:25:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 20, 19, 692494), 1, 2]) securityworker stdout | 2023-08-08 06:25:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:25:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:25:20,108 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:25:21,179 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:25:21,678 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:25:22,179 [199] [DEBUG] [app] Starting request: urn:request:f00e0be3-b760-405b-832e-1eb35fd77dcd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:25:22,180 [197] [DEBUG] [app] Starting request: urn:request:a3bfaad7-d2da-4fa2-8ace-d0f61537faab (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:25:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:25:22,184 [208] [DEBUG] [app] Starting request: urn:request:514e8bfe-aa49-4c5a-8da3-8ea29b5a3a4a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:25:22,185 [208] [DEBUG] [app] Ending request: urn:request:514e8bfe-aa49-4c5a-8da3-8ea29b5a3a4a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:514e8bfe-aa49-4c5a-8da3-8ea29b5a3a4a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:25:22,185 [210] [DEBUG] [app] Starting request: urn:request:49ebeb77-6504-40c6-a0ac-8a2c8299e241 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:25:22,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:25:22,186 [210] [DEBUG] [app] Ending request: urn:request:49ebeb77-6504-40c6-a0ac-8a2c8299e241 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:49ebeb77-6504-40c6-a0ac-8a2c8299e241', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:25:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:25:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:22,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:22,191 [198] [DEBUG] [app] Starting request: urn:request:d0441139-7817-4d91-bb8d-ebbfb110ffb6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:25:22,191 [200] [DEBUG] [app] Starting request: urn:request:1d077ff7-a5bb-409f-8f96-6ab484ceda4a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:25:22,191 [198] [DEBUG] [app] Ending request: urn:request:d0441139-7817-4d91-bb8d-ebbfb110ffb6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d0441139-7817-4d91-bb8d-ebbfb110ffb6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:22,191 [200] [DEBUG] [app] Ending request: urn:request:1d077ff7-a5bb-409f-8f96-6ab484ceda4a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1d077ff7-a5bb-409f-8f96-6ab484ceda4a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:22,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:25:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:25:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:25:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:25:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:25:22,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:25:22,203 [197] [DEBUG] [app] Ending request: urn:request:a3bfaad7-d2da-4fa2-8ace-d0f61537faab (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a3bfaad7-d2da-4fa2-8ace-d0f61537faab', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:25:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:25:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:25:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:25:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:25:22,204 [199] [DEBUG] [app] Ending request: urn:request:f00e0be3-b760-405b-832e-1eb35fd77dcd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f00e0be3-b760-405b-832e-1eb35fd77dcd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:25:22,204 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:25:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:25:22,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:25:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:25:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:25:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:28.874906+00:00 (in 6.580627 seconds) chunkcleanupworker stdout | 2023-08-08 06:25:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:32 UTC)" (scheduled at 2023-08-08 06:25:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:25:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:25:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 22, 294845), True, datetime.datetime(2023, 8, 8, 6, 25, 22, 294845), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:25:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:25:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:25:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:25:24,284 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:25:24,554 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:25:24,564 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:25:24,564 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:25:24,568 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:25:25,306 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:25:25,310 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:25:25,314 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:25:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:25:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:35.371682+00:00 (in 9.999443 seconds) notificationworker stdout | 2023-08-08 06:25:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:35 UTC)" (scheduled at 2023-08-08 06:25:25.371682+00:00) notificationworker stdout | 2023-08-08 06:25:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:25:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 25, 372760), True, datetime.datetime(2023, 8, 8, 6, 25, 25, 372760), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:25:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:25:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:25:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:25:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:47.398551+00:00 (in 21.883626 seconds) exportactionlogsworker stdout | 2023-08-08 06:25:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:25 UTC)" (scheduled at 2023-08-08 06:25:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:25:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:25:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:25:26,159 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:25:26,538 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:25:26,633 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:25:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:25:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:49.687363+00:00 (in 22.796090 seconds) securityworker stdout | 2023-08-08 06:25:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:56 UTC)" (scheduled at 2023-08-08 06:25:26.890741+00:00) securityworker stdout | 2023-08-08 06:25:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:25:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:25:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:25:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:25:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:25:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:25:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:25:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:25:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:25:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:25:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 20, 26, 897657), 1, 2]) securityworker stdout | 2023-08-08 06:25:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 20, 26, 897657), 1, 2]) securityworker stdout | 2023-08-08 06:25:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:25:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:25:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:25:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:25:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:25:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:25:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:25:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:25:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:25:27,649 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:25:27,699 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:25:27,892 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:25:27,966 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:25:27,984 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:25:27,995 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:25:28,169 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:25:28,364 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:25:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:25:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:32.293757+00:00 (in 3.418347 seconds) chunkcleanupworker stdout | 2023-08-08 06:25:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:28 UTC)" (scheduled at 2023-08-08 06:25:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:25:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:25:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:25:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:25:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:39.417660+00:00 (in 9.999427 seconds) gcworker stdout | 2023-08-08 06:25:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:39 UTC)" (scheduled at 2023-08-08 06:25:29.417660+00:00) gcworker stdout | 2023-08-08 06:25:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266329418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:25:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:25:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:25:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:25:29,697 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:25:29,871 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:25:30,858 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:25:31,759 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:25:31,898 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:25:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:25:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:42.293757+00:00 (in 9.999332 seconds) chunkcleanupworker stdout | 2023-08-08 06:25:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:42 UTC)" (scheduled at 2023-08-08 06:25:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:25:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:25:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 32, 294997), True, datetime.datetime(2023, 8, 8, 6, 25, 32, 294997), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:25:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:25:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:25:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:42 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:25:32,325 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:25:32,751 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:25:33,107 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:25:33,482 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:25:33,622 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:25:33,626 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:25:33,629 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:25:33,631 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:25:33,644 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:25:33,648 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:25:33,651 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:25:33,654 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:25:33,658 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:25:34,570 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:25:34,580 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:25:34,580 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:25:34,584 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:25:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:45.371682+00:00 (in 9.999468 seconds) notificationworker stdout | 2023-08-08 06:25:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:45 UTC)" (scheduled at 2023-08-08 06:25:35.371682+00:00) notificationworker stdout | 2023-08-08 06:25:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:25:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 35, 372715), True, datetime.datetime(2023, 8, 8, 6, 25, 35, 372715), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:25:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:25:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:25:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:25:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:54.393627+00:00 (in 17.813256 seconds) namespacegcworker stdout | 2023-08-08 06:25:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:36 UTC)" (scheduled at 2023-08-08 06:25:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:25:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:25:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:25:37,179 [197] [DEBUG] [app] Starting request: urn:request:35787677-a8de-4d54-8f33-4e80b06b5caa (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:25:37,179 [198] [DEBUG] [app] Starting request: urn:request:f0f8fc74-3a2e-4839-ade7-d459928b7542 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:25:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:37,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:37,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:25:37,184 [207] [DEBUG] [app] Starting request: urn:request:b64d6376-72a7-4027-97ee-c83d3b724e7d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:25:37,185 [207] [DEBUG] [app] Ending request: urn:request:b64d6376-72a7-4027-97ee-c83d3b724e7d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b64d6376-72a7-4027-97ee-c83d3b724e7d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:25:37,185 [208] [DEBUG] [app] Starting request: urn:request:6ea4bf77-f1b2-49cc-902c-34c2a8416e0e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:25:37,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:25:37,186 [208] [DEBUG] [app] Ending request: urn:request:6ea4bf77-f1b2-49cc-902c-34c2a8416e0e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6ea4bf77-f1b2-49cc-902c-34c2a8416e0e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:37,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:25:37,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:25:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:37,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:37,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:37,191 [197] [DEBUG] [app] Starting request: urn:request:0b588ce0-913e-4ea7-81bf-3f71690b19a2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:25:37,192 [200] [DEBUG] [app] Starting request: urn:request:44de1754-f1a9-4164-b2f7-5f60922bfcd1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:25:37,192 [197] [DEBUG] [app] Ending request: urn:request:0b588ce0-913e-4ea7-81bf-3f71690b19a2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0b588ce0-913e-4ea7-81bf-3f71690b19a2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:37,192 [200] [DEBUG] [app] Ending request: urn:request:44de1754-f1a9-4164-b2f7-5f60922bfcd1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:44de1754-f1a9-4164-b2f7-5f60922bfcd1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2023-08-08 06:25:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:25:37,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:25:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:25:37,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:37,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:25:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:25:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:25:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:07.197412+00:00 (in 29.999557 seconds) buildlogsarchiver stdout | 2023-08-08 06:25:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:07 UTC)" (scheduled at 2023-08-08 06:25:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:25:37,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:25:37,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:25:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 25, 37, 198275), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:25:37,203 [198] [DEBUG] [app] Ending request: urn:request:f0f8fc74-3a2e-4839-ade7-d459928b7542 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f0f8fc74-3a2e-4839-ade7-d459928b7542', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:25:37,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:25:37,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:25:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:25:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:25:37,204 [197] [DEBUG] [app] Ending request: urn:request:35787677-a8de-4d54-8f33-4e80b06b5caa (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:35787677-a8de-4d54-8f33-4e80b06b5caa', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:25:37,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:25:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:25:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:25:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:25:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:25:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:25:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:25:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:25:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:25:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:25:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:49.417660+00:00 (in 9.999402 seconds) gcworker stdout | 2023-08-08 06:25:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:49 UTC)" (scheduled at 2023-08-08 06:25:39.417660+00:00) gcworker stdout | 2023-08-08 06:25:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266339418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:25:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:25:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:25:39,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:25:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:25:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:52.293757+00:00 (in 9.999448 seconds) chunkcleanupworker stdout | 2023-08-08 06:25:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:52 UTC)" (scheduled at 2023-08-08 06:25:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:25:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:25:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 42, 294779), True, datetime.datetime(2023, 8, 8, 6, 25, 42, 294779), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:25:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:25:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:25:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:52 UTC)" executed successfully builder stdout | 2023-08-08 06:25:44,595 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:25:44,604 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:25:44,604 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:25:44,608 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:25:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:55.371682+00:00 (in 9.999475 seconds) notificationworker stdout | 2023-08-08 06:25:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:55 UTC)" (scheduled at 2023-08-08 06:25:45.371682+00:00) notificationworker stdout | 2023-08-08 06:25:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:25:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 45, 373254), True, datetime.datetime(2023, 8, 8, 6, 25, 45, 373254), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:25:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:25:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:25:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:25:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:25.514341+00:00 (in 38.115278 seconds) exportactionlogsworker stdout | 2023-08-08 06:25:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:47 UTC)" (scheduled at 2023-08-08 06:25:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:25:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:25:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 47, 399435), True, datetime.datetime(2023, 8, 8, 6, 25, 47, 399435), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:25:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:25:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:25:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:25:48,926 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:25:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:25:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:59.417660+00:00 (in 9.999348 seconds) gcworker stdout | 2023-08-08 06:25:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:59 UTC)" (scheduled at 2023-08-08 06:25:49.417660+00:00) gcworker stdout | 2023-08-08 06:25:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266349418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:25:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:25:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:25:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:25:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:25:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:25:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:25:56.890741+00:00 (in 7.202864 seconds) securityworker stdout | 2023-08-08 06:25:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:19 UTC)" (scheduled at 2023-08-08 06:25:49.687363+00:00) securityworker stdout | 2023-08-08 06:25:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:25:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:25:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:25:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:25:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:25:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:25:50,118 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:25:51,215 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:25:51,688 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:25:52,178 [200] [DEBUG] [app] Starting request: urn:request:0d8b58c1-8f7d-40db-a85f-e8dd539f13b9 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:25:52,179 [197] [DEBUG] [app] Starting request: urn:request:bf3e6d40-607f-400c-9521-47f1f9726988 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:25:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:52,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:25:52,185 [203] [DEBUG] [app] Starting request: urn:request:c7bea240-92e1-4cd5-81bb-78a3a9b12885 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:25:52,186 [209] [DEBUG] [app] Starting request: urn:request:1c788609-d959-4318-bc81-9a5bd26893cf (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:25:52,186 [203] [DEBUG] [app] Ending request: urn:request:c7bea240-92e1-4cd5-81bb-78a3a9b12885 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c7bea240-92e1-4cd5-81bb-78a3a9b12885', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:25:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:25:52,186 [209] [DEBUG] [app] Ending request: urn:request:1c788609-d959-4318-bc81-9a5bd26893cf (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1c788609-d959-4318-bc81-9a5bd26893cf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:25:52,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:25:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:25:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:52,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:25:52,191 [200] [DEBUG] [app] Starting request: urn:request:068aed5d-cb2c-4998-aeef-7ddfc01c195e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:25:52,192 [197] [DEBUG] [app] Starting request: urn:request:6200c5b7-f309-4ff4-9294-0fcf80479a8e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:25:52,192 [200] [DEBUG] [app] Ending request: urn:request:068aed5d-cb2c-4998-aeef-7ddfc01c195e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:068aed5d-cb2c-4998-aeef-7ddfc01c195e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:52,192 [197] [DEBUG] [app] Ending request: urn:request:6200c5b7-f309-4ff4-9294-0fcf80479a8e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6200c5b7-f309-4ff4-9294-0fcf80479a8e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:25:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:25:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:25:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:25:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:25:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:52,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:25:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:25:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:25:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:25:52,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:25:52,203 [200] [DEBUG] [app] Ending request: urn:request:0d8b58c1-8f7d-40db-a85f-e8dd539f13b9 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0d8b58c1-8f7d-40db-a85f-e8dd539f13b9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:25:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:25:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:25:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:25:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:25:52,204 [197] [DEBUG] [app] Ending request: urn:request:bf3e6d40-607f-400c-9521-47f1f9726988 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bf3e6d40-607f-400c-9521-47f1f9726988', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:25:52,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:25:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:25:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:25:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:25:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:25:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:02.293757+00:00 (in 9.999472 seconds) chunkcleanupworker stdout | 2023-08-08 06:25:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:02 UTC)" (scheduled at 2023-08-08 06:25:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:25:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:25:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 52, 294878), True, datetime.datetime(2023, 8, 8, 6, 25, 52, 294878), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:25:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:25:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:25:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:25:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:25:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:13.097375+00:00 (in 18.812626 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:25:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:54 UTC)" (scheduled at 2023-08-08 06:25:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:25:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:25:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:25:54,311 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} namespacegcworker stdout | 2023-08-08 06:25:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:25:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:36.579841+00:00 (in 42.185684 seconds) namespacegcworker stdout | 2023-08-08 06:25:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:54 UTC)" (scheduled at 2023-08-08 06:25:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:25:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:25:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 54, 394521), True, datetime.datetime(2023, 8, 8, 6, 25, 54, 394521), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:25:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:25:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:25:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:26:54 UTC)" executed successfully builder stdout | 2023-08-08 06:25:54,619 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:25:54,628 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:25:54,628 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:25:54,633 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:25:55,315 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:25:55,318 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:25:55,322 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:25:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:25:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:02.178530+00:00 (in 6.806329 seconds) notificationworker stdout | 2023-08-08 06:25:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:05 UTC)" (scheduled at 2023-08-08 06:25:55.371682+00:00) notificationworker stdout | 2023-08-08 06:25:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:25:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 25, 55, 372726), True, datetime.datetime(2023, 8, 8, 6, 25, 55, 372726), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:25:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:25:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:25:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:25:56,186 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:25:56,572 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:25:56,663 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:25:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:25:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:19.687363+00:00 (in 22.796138 seconds) securityworker stdout | 2023-08-08 06:25:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:26 UTC)" (scheduled at 2023-08-08 06:25:56.890741+00:00) securityworker stdout | 2023-08-08 06:25:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:25:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:25:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:25:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:25:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:25:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:25:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:25:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:25:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:25:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:25:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 20, 56, 896716), 1, 2]) securityworker stdout | 2023-08-08 06:25:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 20, 56, 896716), 1, 2]) securityworker stdout | 2023-08-08 06:25:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:25:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:25:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:25:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:25:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:25:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:25:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:25:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:25:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:25:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:25:57,660 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:25:57,707 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:25:57,904 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:25:57,982 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:25:57,997 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:25:58,008 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:25:58,188 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:25:58,402 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:25:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:25:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:09.417660+00:00 (in 9.999359 seconds) gcworker stdout | 2023-08-08 06:25:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:09 UTC)" (scheduled at 2023-08-08 06:25:59.417660+00:00) gcworker stdout | 2023-08-08 06:25:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266359418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:25:59,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:25:59,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:25:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:25:59,722 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:25:59,894 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:26:00,890 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:26:01,797 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:26:01,936 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} notificationworker stdout | 2023-08-08 06:26:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:26:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:05.371682+00:00 (in 3.192645 seconds) notificationworker stdout | 2023-08-08 06:26:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:02 UTC)" (scheduled at 2023-08-08 06:26:02.178530+00:00) notificationworker stdout | 2023-08-08 06:26:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:26:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:26:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:26:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:12.293757+00:00 (in 9.999323 seconds) chunkcleanupworker stdout | 2023-08-08 06:26:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:12 UTC)" (scheduled at 2023-08-08 06:26:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:26:02,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:26:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 2, 295049), True, datetime.datetime(2023, 8, 8, 6, 26, 2, 295049), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:26:02,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:26:02,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:26:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:12 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:26:02,363 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:26:02,788 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:26:03,145 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:26:03,521 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:26:03,634 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:26:03,639 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:26:03,643 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:26:03,648 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:26:03,655 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:26:03,660 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:26:03,664 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:26:03,667 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:26:03,670 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:26:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:26:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:09.793658+00:00 (in 5.605716 seconds) repositorygcworker stdout | 2023-08-08 06:26:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:04 UTC)" (scheduled at 2023-08-08 06:26:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:26:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:26:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:04 UTC)" executed successfully builder stdout | 2023-08-08 06:26:04,643 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:26:04,653 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:26:04,653 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:26:04,657 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:26:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:15.371682+00:00 (in 9.999418 seconds) notificationworker stdout | 2023-08-08 06:26:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:15 UTC)" (scheduled at 2023-08-08 06:26:05.371682+00:00) notificationworker stdout | 2023-08-08 06:26:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:26:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 5, 372842), True, datetime.datetime(2023, 8, 8, 6, 26, 5, 372842), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:26:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:26:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:26:07,179 [200] [DEBUG] [app] Starting request: urn:request:a199a265-2abb-42af-b2e4-6e957bb286b7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:26:07,180 [197] [DEBUG] [app] Starting request: urn:request:f8b8a80a-8cde-4015-b97c-8d5422797b03 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:26:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:07,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:26:07,185 [210] [DEBUG] [app] Starting request: urn:request:b8b3c5e5-9161-43ba-9b69-73bf6068be81 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:26:07,186 [207] [DEBUG] [app] Starting request: urn:request:3d0af2b4-1bd8-49bf-89db-160fa57ec3b6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:26:07,186 [210] [DEBUG] [app] Ending request: urn:request:b8b3c5e5-9161-43ba-9b69-73bf6068be81 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b8b3c5e5-9161-43ba-9b69-73bf6068be81', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:26:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:26:07,186 [207] [DEBUG] [app] Ending request: urn:request:3d0af2b4-1bd8-49bf-89db-160fa57ec3b6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3d0af2b4-1bd8-49bf-89db-160fa57ec3b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:26:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:26:07,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:07,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:07,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:07,192 [197] [DEBUG] [app] Starting request: urn:request:bbf43074-f506-45d7-9ab8-7f1d1d914b8e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:26:07,193 [197] [DEBUG] [app] Ending request: urn:request:bbf43074-f506-45d7-9ab8-7f1d1d914b8e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bbf43074-f506-45d7-9ab8-7f1d1d914b8e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:26:07,193 [199] [DEBUG] [app] Starting request: urn:request:4eeff6e8-aa76-47ef-b0a0-7a907abdf4ae (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:26:07,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:07,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:07,194 [199] [DEBUG] [app] Ending request: urn:request:4eeff6e8-aa76-47ef-b0a0-7a907abdf4ae (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4eeff6e8-aa76-47ef-b0a0-7a907abdf4ae', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:26:07,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:26:07,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:07,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:07,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:26:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:26:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:37.197412+00:00 (in 29.999430 seconds) buildlogsarchiver stdout | 2023-08-08 06:26:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:37 UTC)" (scheduled at 2023-08-08 06:26:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:26:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 26, 7, 198393), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:26:07,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:26:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:26:07,204 [197] [DEBUG] [app] Ending request: urn:request:f8b8a80a-8cde-4015-b97c-8d5422797b03 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f8b8a80a-8cde-4015-b97c-8d5422797b03', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:26:07,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:26:07,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:26:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:26:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:26:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:26:07,206 [200] [DEBUG] [app] Ending request: urn:request:a199a265-2abb-42af-b2e4-6e957bb286b7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a199a265-2abb-42af-b2e4-6e957bb286b7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:26:07,207 [200] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:26:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:26:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.028) gunicorn-web stdout | 2023-08-08 06:26:07,207 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:26:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:26:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:26:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:26:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:26:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:26:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:19.417660+00:00 (in 9.999473 seconds) gcworker stdout | 2023-08-08 06:26:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:19 UTC)" (scheduled at 2023-08-08 06:26:09.417660+00:00) gcworker stdout | 2023-08-08 06:26:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266369419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:26:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:26:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:26:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:26:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:26:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:04.187429+00:00 (in 54.393231 seconds) repositorygcworker stdout | 2023-08-08 06:26:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:09 UTC)" (scheduled at 2023-08-08 06:26:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:26:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:26:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 9, 794867), True, datetime.datetime(2023, 8, 8, 6, 26, 9, 794867), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:26:09,804 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:26:09,804 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:26:09,805 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:26:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:26:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:22.293757+00:00 (in 9.999338 seconds) chunkcleanupworker stdout | 2023-08-08 06:26:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:22 UTC)" (scheduled at 2023-08-08 06:26:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:26:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:26:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 12, 294903), True, datetime.datetime(2023, 8, 8, 6, 26, 12, 294903), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:26:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:26:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:26:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:26:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:26:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:31.283850+00:00 (in 18.185936 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:26:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:13 UTC)" (scheduled at 2023-08-08 06:26:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:26:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:26:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 13, 98409), True, datetime.datetime(2023, 8, 8, 6, 26, 13, 98409), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:26:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:26:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:26:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:13 UTC)" executed successfully builder stdout | 2023-08-08 06:26:14,668 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:26:14,679 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:26:14,679 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:26:14,684 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:26:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:25.371682+00:00 (in 9.999360 seconds) notificationworker stdout | 2023-08-08 06:26:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:25 UTC)" (scheduled at 2023-08-08 06:26:15.371682+00:00) notificationworker stdout | 2023-08-08 06:26:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:26:15,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 15, 373029), True, datetime.datetime(2023, 8, 8, 6, 26, 15, 373029), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:26:15,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:26:15,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:15,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:26:18,935 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:26:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:26:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:29.417660+00:00 (in 9.999481 seconds) gcworker stdout | 2023-08-08 06:26:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:29 UTC)" (scheduled at 2023-08-08 06:26:19.417660+00:00) gcworker stdout | 2023-08-08 06:26:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266379418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:26:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:26:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:26:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:26:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:26:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:26.890741+00:00 (in 7.202731 seconds) securityworker stdout | 2023-08-08 06:26:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:49 UTC)" (scheduled at 2023-08-08 06:26:19.687363+00:00) securityworker stdout | 2023-08-08 06:26:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:26:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:26:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:26:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:26:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:26:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:26:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 21, 19, 692314), 1, 2]) securityworker stdout | 2023-08-08 06:26:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:19,705 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 21, 19, 692314), 1, 2]) securityworker stdout | 2023-08-08 06:26:19,706 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:26:19,707 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:26:20,140 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:26:21,231 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:26:21,702 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:26:22,179 [197] [DEBUG] [app] Starting request: urn:request:1d286f6e-298a-41b9-a54a-e5f3fbbb0036 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:26:22,179 [199] [DEBUG] [app] Starting request: urn:request:a442ebae-957b-48c6-8749-a4f170fc05a7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:26:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:26:22,185 [203] [DEBUG] [app] Starting request: urn:request:65426287-f963-400c-a993-536b1edb87b7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:26:22,185 [207] [DEBUG] [app] Starting request: urn:request:5f7f09ce-ee54-400c-bf0b-ee8b6a5b7638 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:26:22,185 [203] [DEBUG] [app] Ending request: urn:request:65426287-f963-400c-a993-536b1edb87b7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:65426287-f963-400c-a993-536b1edb87b7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:26:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:26:22,186 [207] [DEBUG] [app] Ending request: urn:request:5f7f09ce-ee54-400c-bf0b-ee8b6a5b7638 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5f7f09ce-ee54-400c-bf0b-ee8b6a5b7638', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:26:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:26:22,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:22,190 [198] [DEBUG] [app] Starting request: urn:request:529ad500-6d5e-409f-81f9-02ca8d473547 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:26:22,191 [198] [DEBUG] [app] Ending request: urn:request:529ad500-6d5e-409f-81f9-02ca8d473547 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:529ad500-6d5e-409f-81f9-02ca8d473547', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:26:22,191 [200] [DEBUG] [app] Starting request: urn:request:d578c8ff-9dc5-4263-8104-c5ed6648f1d3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:26:22,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:22,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:22,191 [200] [DEBUG] [app] Ending request: urn:request:d578c8ff-9dc5-4263-8104-c5ed6648f1d3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d578c8ff-9dc5-4263-8104-c5ed6648f1d3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:26:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:26:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:26:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:26:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:26:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:26:22,203 [199] [DEBUG] [app] Ending request: urn:request:a442ebae-957b-48c6-8749-a4f170fc05a7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a442ebae-957b-48c6-8749-a4f170fc05a7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:26:22,203 [197] [DEBUG] [app] Ending request: urn:request:1d286f6e-298a-41b9-a54a-e5f3fbbb0036 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1d286f6e-298a-41b9-a54a-e5f3fbbb0036', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:26:22,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:26:22,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:26:22,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:26:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:26:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:26:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:26:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:26:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:26:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:26:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:28.874906+00:00 (in 6.580609 seconds) chunkcleanupworker stdout | 2023-08-08 06:26:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:32 UTC)" (scheduled at 2023-08-08 06:26:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:26:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:26:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 22, 294797), True, datetime.datetime(2023, 8, 8, 6, 26, 22, 294797), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:26:22,305 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:26:22,305 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:26:22,305 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:26:24,327 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:26:24,694 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:26:24,703 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:26:24,703 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:26:24,707 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:26:25,326 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:26:25,328 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:26:25,331 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:26:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:26:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:35.371682+00:00 (in 9.999453 seconds) notificationworker stdout | 2023-08-08 06:26:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:35 UTC)" (scheduled at 2023-08-08 06:26:25.371682+00:00) notificationworker stdout | 2023-08-08 06:26:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:26:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 25, 372804), True, datetime.datetime(2023, 8, 8, 6, 26, 25, 372804), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:26:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:26:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:26:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:26:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:47.398551+00:00 (in 21.883700 seconds) exportactionlogsworker stdout | 2023-08-08 06:26:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:25 UTC)" (scheduled at 2023-08-08 06:26:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:26:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:26:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:26:26,223 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:26:26,608 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:26:26,672 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:26:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:26:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:49.687363+00:00 (in 22.796109 seconds) securityworker stdout | 2023-08-08 06:26:26,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:56 UTC)" (scheduled at 2023-08-08 06:26:26.890741+00:00) securityworker stdout | 2023-08-08 06:26:26,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:26:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:26:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:26:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:26:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:26:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:26:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:26:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:26:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:26:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:26:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 21, 26, 896703), 1, 2]) securityworker stdout | 2023-08-08 06:26:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 21, 26, 896703), 1, 2]) securityworker stdout | 2023-08-08 06:26:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:26:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:26:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:26:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:26:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:26:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:26:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:26:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:26:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:26:27,689 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:26:27,716 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:26:27,916 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:26:27,997 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:26:28,009 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:26:28,021 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:26:28,225 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:26:28,439 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:26:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:26:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:32.293757+00:00 (in 3.418341 seconds) chunkcleanupworker stdout | 2023-08-08 06:26:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:28 UTC)" (scheduled at 2023-08-08 06:26:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:26:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:26:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:26:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:26:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:39.417660+00:00 (in 9.999483 seconds) gcworker stdout | 2023-08-08 06:26:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:39 UTC)" (scheduled at 2023-08-08 06:26:29.417660+00:00) gcworker stdout | 2023-08-08 06:26:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266389418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:26:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:26:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:26:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:26:29,760 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:26:29,913 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:26:30,927 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} securityscanningnotificationworker stdout | 2023-08-08 06:26:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:26:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:54.284093+00:00 (in 22.999663 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:26:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:31:31 UTC)" (scheduled at 2023-08-08 06:26:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:26:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 26, 31, 284959), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:26:31,292 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 26, 31, 284959), True, datetime.datetime(2023, 8, 8, 6, 26, 31, 284959), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:26:31,294 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 26, 31, 284959), True, datetime.datetime(2023, 8, 8, 6, 26, 31, 284959), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 26, 31, 284959), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:26:31,295 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:26:31,295 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:31:31 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:26:31,816 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:26:31,945 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:26:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:26:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:42.293757+00:00 (in 9.999433 seconds) chunkcleanupworker stdout | 2023-08-08 06:26:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:42 UTC)" (scheduled at 2023-08-08 06:26:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:26:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:26:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 32, 294976), True, datetime.datetime(2023, 8, 8, 6, 26, 32, 294976), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:26:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:26:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:26:32,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:42 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:26:32,386 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:26:32,826 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:26:33,156 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:26:33,559 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:26:33,644 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:26:33,646 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:26:33,651 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:26:33,655 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:26:33,664 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:26:33,671 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:26:33,674 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:26:33,677 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:26:33,680 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:26:34,717 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:26:34,726 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:26:34,727 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:26:34,731 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:26:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:45.371682+00:00 (in 9.999386 seconds) notificationworker stdout | 2023-08-08 06:26:35,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:45 UTC)" (scheduled at 2023-08-08 06:26:35.371682+00:00) notificationworker stdout | 2023-08-08 06:26:35,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:26:35,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 35, 373368), True, datetime.datetime(2023, 8, 8, 6, 26, 35, 373368), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:26:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:26:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:26:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:26:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:54.393627+00:00 (in 17.813218 seconds) namespacegcworker stdout | 2023-08-08 06:26:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:36 UTC)" (scheduled at 2023-08-08 06:26:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:26:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:26:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:26:37,179 [197] [DEBUG] [app] Starting request: urn:request:0d157972-4ba3-4e41-9c48-9e843a7e8ada (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:26:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:37,181 [198] [DEBUG] [app] Starting request: urn:request:0cabac2b-9cc1-4453-8075-20d4a00e11a1 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:26:37,182 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:26:37,184 [206] [DEBUG] [app] Starting request: urn:request:dccdc566-0fd0-46d4-9dd7-6da1830cb709 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:26:37,185 [206] [DEBUG] [app] Ending request: urn:request:dccdc566-0fd0-46d4-9dd7-6da1830cb709 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:dccdc566-0fd0-46d4-9dd7-6da1830cb709', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:26:37,185 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:26:37,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:26:37,186 [206] [DEBUG] [app] Starting request: urn:request:067cf891-a5ad-45c6-bd30-43ee91cd0f66 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:26:37,186 [206] [DEBUG] [app] Ending request: urn:request:067cf891-a5ad-45c6-bd30-43ee91cd0f66 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:067cf891-a5ad-45c6-bd30-43ee91cd0f66', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:26:37,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.001) gunicorn-web stdout | 2023-08-08 06:26:37,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:37,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:37,190 [198] [DEBUG] [app] Starting request: urn:request:1e6a2510-fbf8-4133-a6aa-33d317a51edf (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:26:37,190 [198] [DEBUG] [app] Ending request: urn:request:1e6a2510-fbf8-4133-a6aa-33d317a51edf (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1e6a2510-fbf8-4133-a6aa-33d317a51edf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:26:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:37,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:37,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:26:37,191 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:37,192 [199] [DEBUG] [app] Starting request: urn:request:2bc97c5f-2048-42e3-a675-ad9af79ddf49 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:26:37,193 [199] [DEBUG] [app] Ending request: urn:request:2bc97c5f-2048-42e3-a675-ad9af79ddf49 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2bc97c5f-2048-42e3-a675-ad9af79ddf49', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:26:37,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:37,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:37,194 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:26:37,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:26:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:26:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:07.197412+00:00 (in 29.999553 seconds) buildlogsarchiver stdout | 2023-08-08 06:26:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:07 UTC)" (scheduled at 2023-08-08 06:26:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:26:37,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:26:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 26, 37, 198702), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:26:37,202 [197] [DEBUG] [app] Ending request: urn:request:0d157972-4ba3-4e41-9c48-9e843a7e8ada (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0d157972-4ba3-4e41-9c48-9e843a7e8ada', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:26:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:26:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:26:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:26:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:26:37,204 [198] [DEBUG] [app] Ending request: urn:request:0cabac2b-9cc1-4453-8075-20d4a00e11a1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0cabac2b-9cc1-4453-8075-20d4a00e11a1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:26:37,204 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:26:37,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:26:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:26:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:26:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:26:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:26:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:26:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:26:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:26:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:26:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:49.417660+00:00 (in 9.999489 seconds) gcworker stdout | 2023-08-08 06:26:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:49 UTC)" (scheduled at 2023-08-08 06:26:39.417660+00:00) gcworker stdout | 2023-08-08 06:26:39,418 [61] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2023-08-08 06:26:39,425 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266399424, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:26:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:26:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:26:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:26:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:26:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:52.293757+00:00 (in 9.999427 seconds) chunkcleanupworker stdout | 2023-08-08 06:26:42,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:52 UTC)" (scheduled at 2023-08-08 06:26:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:26:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:26:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 42, 295343), True, datetime.datetime(2023, 8, 8, 6, 26, 42, 295343), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:26:42,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:26:42,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:26:42,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:52 UTC)" executed successfully builder stdout | 2023-08-08 06:26:44,732 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:26:44,742 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:26:44,742 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:26:44,747 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:26:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:55.371682+00:00 (in 9.999357 seconds) notificationworker stdout | 2023-08-08 06:26:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:55 UTC)" (scheduled at 2023-08-08 06:26:45.371682+00:00) notificationworker stdout | 2023-08-08 06:26:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:26:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 45, 372904), True, datetime.datetime(2023, 8, 8, 6, 26, 45, 372904), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:26:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:26:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:26:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:26:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:25.514341+00:00 (in 38.115337 seconds) exportactionlogsworker stdout | 2023-08-08 06:26:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:47 UTC)" (scheduled at 2023-08-08 06:26:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:26:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:26:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 47, 399521), True, datetime.datetime(2023, 8, 8, 6, 26, 47, 399521), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:26:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:26:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:26:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:26:48,973 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:26:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:26:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:59.417660+00:00 (in 9.999503 seconds) gcworker stdout | 2023-08-08 06:26:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:59 UTC)" (scheduled at 2023-08-08 06:26:49.417660+00:00) gcworker stdout | 2023-08-08 06:26:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266409418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:26:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:26:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:26:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:26:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:26:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:26:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:26:56.890741+00:00 (in 7.202859 seconds) securityworker stdout | 2023-08-08 06:26:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:19 UTC)" (scheduled at 2023-08-08 06:26:49.687363+00:00) securityworker stdout | 2023-08-08 06:26:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:26:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:26:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:26:49,691 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:26:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:26:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:26:50,177 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:26:51,268 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:26:51,712 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:26:52,179 [200] [DEBUG] [app] Starting request: urn:request:db037400-1fad-4db9-91a8-cc7ec91af312 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:26:52,180 [199] [DEBUG] [app] Starting request: urn:request:55c13b32-422f-4502-a8db-2b0b283f04dd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:26:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:52,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:52,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:26:52,185 [207] [DEBUG] [app] Starting request: urn:request:ef1184e4-677b-46b1-a737-6123eef18d47 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:26:52,186 [203] [DEBUG] [app] Starting request: urn:request:f9c82a88-89a5-4afb-bd0a-b941e108f470 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:26:52,186 [207] [DEBUG] [app] Ending request: urn:request:ef1184e4-677b-46b1-a737-6123eef18d47 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ef1184e4-677b-46b1-a737-6123eef18d47', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:26:52,186 [203] [DEBUG] [app] Ending request: urn:request:f9c82a88-89a5-4afb-bd0a-b941e108f470 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f9c82a88-89a5-4afb-bd0a-b941e108f470', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:26:52,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:26:52,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:26:52,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:26:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:52,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:26:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:52,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:26:52,192 [199] [DEBUG] [app] Starting request: urn:request:ba023dd6-7b0a-48c7-ada9-eff8fdafdfa1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:26:52,192 [199] [DEBUG] [app] Ending request: urn:request:ba023dd6-7b0a-48c7-ada9-eff8fdafdfa1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ba023dd6-7b0a-48c7-ada9-eff8fdafdfa1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:26:52,192 [198] [DEBUG] [app] Starting request: urn:request:c2bd09fd-e319-4285-a4ea-da5b3e9cbbfe (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:26:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:26:52,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:52,193 [198] [DEBUG] [app] Ending request: urn:request:c2bd09fd-e319-4285-a4ea-da5b3e9cbbfe (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c2bd09fd-e319-4285-a4ea-da5b3e9cbbfe', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:26:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:26:52,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:26:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:26:52,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:26:52,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:26:52,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:26:52,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:26:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:26:52,204 [200] [DEBUG] [app] Ending request: urn:request:db037400-1fad-4db9-91a8-cc7ec91af312 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:db037400-1fad-4db9-91a8-cc7ec91af312', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:26:52,204 [199] [DEBUG] [app] Ending request: urn:request:55c13b32-422f-4502-a8db-2b0b283f04dd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:55c13b32-422f-4502-a8db-2b0b283f04dd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:26:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:26:52,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:26:52,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:26:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:26:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 06:26:52,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:26:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:26:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:26:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:26:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:02.293757+00:00 (in 9.999465 seconds) chunkcleanupworker stdout | 2023-08-08 06:26:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:02 UTC)" (scheduled at 2023-08-08 06:26:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:26:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:26:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 52, 294784), True, datetime.datetime(2023, 8, 8, 6, 26, 52, 294784), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:26:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:26:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:26:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:26:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:26:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:13.097375+00:00 (in 18.812581 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:26:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:54 UTC)" (scheduled at 2023-08-08 06:26:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:26:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:26:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:26:54,365 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} namespacegcworker stdout | 2023-08-08 06:26:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:26:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:36.579841+00:00 (in 42.185636 seconds) namespacegcworker stdout | 2023-08-08 06:26:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:54 UTC)" (scheduled at 2023-08-08 06:26:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:26:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:26:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 54, 394671), True, datetime.datetime(2023, 8, 8, 6, 26, 54, 394671), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:26:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:26:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:26:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:27:54 UTC)" executed successfully builder stdout | 2023-08-08 06:26:54,756 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:26:54,766 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:26:54,766 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:26:54,770 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:26:55,335 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:26:55,338 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:26:55,341 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:26:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:26:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:02.178530+00:00 (in 6.806351 seconds) notificationworker stdout | 2023-08-08 06:26:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:05 UTC)" (scheduled at 2023-08-08 06:26:55.371682+00:00) notificationworker stdout | 2023-08-08 06:26:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:26:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 26, 55, 372655), True, datetime.datetime(2023, 8, 8, 6, 26, 55, 372655), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:26:55,379 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:26:55,379 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:26:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:26:56,238 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:26:56,636 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:26:56,679 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:26:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:26:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:19.687363+00:00 (in 22.796064 seconds) securityworker stdout | 2023-08-08 06:26:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:26 UTC)" (scheduled at 2023-08-08 06:26:56.890741+00:00) securityworker stdout | 2023-08-08 06:26:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:26:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:26:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:26:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:26:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:26:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:26:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:26:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:26:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:26:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:26:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 21, 56, 896329), 1, 2]) securityworker stdout | 2023-08-08 06:26:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 21, 56, 896329), 1, 2]) securityworker stdout | 2023-08-08 06:26:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:26:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:26:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:26:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:26:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:26:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:26:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:26:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:26:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:26:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:26:57,720 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:26:57,725 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:26:57,928 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:26:58,011 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:26:58,021 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:26:58,033 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:26:58,262 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:26:58,473 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:26:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:26:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:09.417660+00:00 (in 9.999506 seconds) gcworker stdout | 2023-08-08 06:26:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:09 UTC)" (scheduled at 2023-08-08 06:26:59.417660+00:00) gcworker stdout | 2023-08-08 06:26:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266419418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:26:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:26:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:26:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:26:59,777 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:26:59,950 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:27:00,964 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:27:01,851 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:27:01,980 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} notificationworker stdout | 2023-08-08 06:27:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:27:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:05.371682+00:00 (in 3.192582 seconds) notificationworker stdout | 2023-08-08 06:27:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:02 UTC)" (scheduled at 2023-08-08 06:27:02.178530+00:00) notificationworker stdout | 2023-08-08 06:27:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:27:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:27:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:27:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:12.293757+00:00 (in 9.999483 seconds) chunkcleanupworker stdout | 2023-08-08 06:27:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:12 UTC)" (scheduled at 2023-08-08 06:27:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:27:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:27:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 2, 294849), True, datetime.datetime(2023, 8, 8, 6, 27, 2, 294849), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:27:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:27:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:27:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:12 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:27:02,404 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:27:02,838 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:27:03,178 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:27:03,583 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:27:03,653 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:27:03,656 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:27:03,659 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:27:03,664 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:27:03,672 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:27:03,680 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:27:03,683 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:27:03,687 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:27:03,692 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:27:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:27:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:09.793658+00:00 (in 5.605687 seconds) repositorygcworker stdout | 2023-08-08 06:27:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:04 UTC)" (scheduled at 2023-08-08 06:27:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:27:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:27:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:04 UTC)" executed successfully builder stdout | 2023-08-08 06:27:04,772 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:27:04,782 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:27:04,782 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:27:04,786 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:27:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:15.371682+00:00 (in 9.999377 seconds) notificationworker stdout | 2023-08-08 06:27:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:15 UTC)" (scheduled at 2023-08-08 06:27:05.371682+00:00) notificationworker stdout | 2023-08-08 06:27:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:27:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 5, 372704), True, datetime.datetime(2023, 8, 8, 6, 27, 5, 372704), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:27:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:27:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:27:07,178 [197] [DEBUG] [app] Starting request: urn:request:2738daf7-f762-4705-9cf2-6d33dfd1e50d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:27:07,178 [200] [DEBUG] [app] Starting request: urn:request:9ab82fed-2a2f-434b-8109-45a4646938a5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:27:07,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:07,182 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:27:07,184 [203] [DEBUG] [app] Starting request: urn:request:eeab920d-00d9-41bd-99c1-fe5203f26ae7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:27:07,184 [203] [DEBUG] [app] Ending request: urn:request:eeab920d-00d9-41bd-99c1-fe5203f26ae7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:eeab920d-00d9-41bd-99c1-fe5203f26ae7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:27:07,184 [207] [DEBUG] [app] Starting request: urn:request:ca7723fd-5908-4b12-aef6-9d7db5487ddd (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:27:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:27:07,185 [207] [DEBUG] [app] Ending request: urn:request:ca7723fd-5908-4b12-aef6-9d7db5487ddd (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ca7723fd-5908-4b12-aef6-9d7db5487ddd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:27:07,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:27:07,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:27:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:07,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:07,191 [200] [DEBUG] [app] Starting request: urn:request:36e9b187-33ce-426f-b74d-4e943409ad60 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:27:07,191 [199] [DEBUG] [app] Starting request: urn:request:b6ecab31-0245-467f-9111-2877fe6a6473 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:27:07,191 [200] [DEBUG] [app] Ending request: urn:request:36e9b187-33ce-426f-b74d-4e943409ad60 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:36e9b187-33ce-426f-b74d-4e943409ad60', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:27:07,191 [199] [DEBUG] [app] Ending request: urn:request:b6ecab31-0245-467f-9111-2877fe6a6473 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b6ecab31-0245-467f-9111-2877fe6a6473', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:27:07,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:27:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:27:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:27:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:27:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:27:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:27:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:27:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:37.197412+00:00 (in 29.999483 seconds) buildlogsarchiver stdout | 2023-08-08 06:27:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:37 UTC)" (scheduled at 2023-08-08 06:27:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:27:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 27, 7, 198421), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:27:07,203 [200] [DEBUG] [app] Ending request: urn:request:9ab82fed-2a2f-434b-8109-45a4646938a5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9ab82fed-2a2f-434b-8109-45a4646938a5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:27:07,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:27:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 06:27:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:27:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:27:07,205 [197] [DEBUG] [app] Ending request: urn:request:2738daf7-f762-4705-9cf2-6d33dfd1e50d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2738daf7-f762-4705-9cf2-6d33dfd1e50d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:27:07,206 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:27:07,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:27:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:27:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.029) buildlogsarchiver stdout | 2023-08-08 06:27:07,207 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:27:07,208 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:27:07,209 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:27:07,212 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:27:07,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:27:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:27:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:19.417660+00:00 (in 9.999448 seconds) gcworker stdout | 2023-08-08 06:27:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:19 UTC)" (scheduled at 2023-08-08 06:27:09.417660+00:00) gcworker stdout | 2023-08-08 06:27:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266429418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:27:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:27:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:27:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:27:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:27:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:04.187429+00:00 (in 54.393264 seconds) repositorygcworker stdout | 2023-08-08 06:27:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:09 UTC)" (scheduled at 2023-08-08 06:27:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:27:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:27:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 9, 794643), True, datetime.datetime(2023, 8, 8, 6, 27, 9, 794643), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:27:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:27:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:27:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:27:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:27:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:22.293757+00:00 (in 9.999467 seconds) chunkcleanupworker stdout | 2023-08-08 06:27:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:22 UTC)" (scheduled at 2023-08-08 06:27:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:27:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:27:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 12, 294816), True, datetime.datetime(2023, 8, 8, 6, 27, 12, 294816), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:27:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:27:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:27:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:27:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:27:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:54.284093+00:00 (in 41.186159 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:27:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:13 UTC)" (scheduled at 2023-08-08 06:27:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:27:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:27:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 13, 98509), True, datetime.datetime(2023, 8, 8, 6, 27, 13, 98509), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:27:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:27:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:27:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:13 UTC)" executed successfully builder stdout | 2023-08-08 06:27:14,790 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:27:14,799 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:27:14,799 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:27:14,803 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:27:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:25.371682+00:00 (in 9.999514 seconds) notificationworker stdout | 2023-08-08 06:27:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:25 UTC)" (scheduled at 2023-08-08 06:27:15.371682+00:00) notificationworker stdout | 2023-08-08 06:27:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:27:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 15, 372528), True, datetime.datetime(2023, 8, 8, 6, 27, 15, 372528), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:27:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:27:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:27:19,010 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:27:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:27:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:29.417660+00:00 (in 9.999454 seconds) gcworker stdout | 2023-08-08 06:27:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:29 UTC)" (scheduled at 2023-08-08 06:27:19.417660+00:00) gcworker stdout | 2023-08-08 06:27:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266439418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:27:19,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:27:19,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:27:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:27:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:27:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:26.890741+00:00 (in 7.202872 seconds) securityworker stdout | 2023-08-08 06:27:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:49 UTC)" (scheduled at 2023-08-08 06:27:19.687363+00:00) securityworker stdout | 2023-08-08 06:27:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:27:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:27:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:27:19,691 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:27:19,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:27:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:27:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 22, 19, 691662), 1, 2]) securityworker stdout | 2023-08-08 06:27:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 22, 19, 691662), 1, 2]) securityworker stdout | 2023-08-08 06:27:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:27:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:27:20,215 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:27:21,286 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:27:21,749 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:27:22,178 [198] [DEBUG] [app] Starting request: urn:request:6aea74b8-5a81-41c6-98ce-290907cbfde4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:27:22,178 [199] [DEBUG] [app] Starting request: urn:request:7a28263f-9192-4434-bd16-489f74671c8a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:27:22,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:22,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:22,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:27:22,184 [203] [DEBUG] [app] Starting request: urn:request:8d393a9f-cc6d-4b6e-9f51-0dce4d0e25a7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:27:22,185 [203] [DEBUG] [app] Ending request: urn:request:8d393a9f-cc6d-4b6e-9f51-0dce4d0e25a7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8d393a9f-cc6d-4b6e-9f51-0dce4d0e25a7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:27:22,185 [209] [DEBUG] [app] Starting request: urn:request:5e77829a-d444-4280-a2e0-49b38b027554 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:27:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:27:22,185 [209] [DEBUG] [app] Ending request: urn:request:5e77829a-d444-4280-a2e0-49b38b027554 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5e77829a-d444-4280-a2e0-49b38b027554', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:27:22,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:27:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:27:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:27:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:22,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:22,190 [200] [DEBUG] [app] Starting request: urn:request:b836b440-567d-430a-8650-7accb1010303 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:27:22,191 [200] [DEBUG] [app] Ending request: urn:request:b836b440-567d-430a-8650-7accb1010303 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b836b440-567d-430a-8650-7accb1010303', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:27:22,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:27:22,191 [197] [DEBUG] [app] Starting request: urn:request:9275a80f-4cf6-46ae-9b28-31f8627a9c44 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:27:22,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:22,192 [197] [DEBUG] [app] Ending request: urn:request:9275a80f-4cf6-46ae-9b28-31f8627a9c44 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9275a80f-4cf6-46ae-9b28-31f8627a9c44', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:27:22,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:27:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:27:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:27:22,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:27:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:27:22,202 [198] [DEBUG] [app] Ending request: urn:request:6aea74b8-5a81-41c6-98ce-290907cbfde4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6aea74b8-5a81-41c6-98ce-290907cbfde4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:27:22,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:27:22,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:27:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:27:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:27:22,203 [199] [DEBUG] [app] Ending request: urn:request:7a28263f-9192-4434-bd16-489f74671c8a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7a28263f-9192-4434-bd16-489f74671c8a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:27:22,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:27:22,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:27:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:27:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:27:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:27:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:28.874906+00:00 (in 6.580605 seconds) chunkcleanupworker stdout | 2023-08-08 06:27:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:32 UTC)" (scheduled at 2023-08-08 06:27:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:27:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:27:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 22, 294845), True, datetime.datetime(2023, 8, 8, 6, 27, 22, 294845), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:27:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:27:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:27:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:27:24,388 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:27:24,804 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:27:24,813 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:27:24,814 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:27:24,818 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:27:25,344 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:27:25,347 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:27:25,352 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:27:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:27:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:35.371682+00:00 (in 9.999482 seconds) notificationworker stdout | 2023-08-08 06:27:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:35 UTC)" (scheduled at 2023-08-08 06:27:25.371682+00:00) notificationworker stdout | 2023-08-08 06:27:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:27:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 25, 372699), True, datetime.datetime(2023, 8, 8, 6, 27, 25, 372699), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:27:25,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:27:25,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:25,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:27:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:27:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:47.398551+00:00 (in 21.883673 seconds) exportactionlogsworker stdout | 2023-08-08 06:27:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:25 UTC)" (scheduled at 2023-08-08 06:27:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:27:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:27:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:27:26,275 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:27:26,672 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:27:26,686 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:27:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:27:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:49.687363+00:00 (in 22.796074 seconds) securityworker stdout | 2023-08-08 06:27:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:56 UTC)" (scheduled at 2023-08-08 06:27:26.890741+00:00) securityworker stdout | 2023-08-08 06:27:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:27:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:27:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:27:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:27:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:27:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:27:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:27:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:27:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:27:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:27:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 22, 26, 896255), 1, 2]) securityworker stdout | 2023-08-08 06:27:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 22, 26, 896255), 1, 2]) securityworker stdout | 2023-08-08 06:27:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:27:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:27:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:27:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:27:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:27:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:27:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:27:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:27:56 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:27:27,732 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:27:27,736 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:27:27,942 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:27:28,030 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:27:28,036 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:27:28,046 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:27:28,289 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:27:28,497 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:27:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:27:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:32.293757+00:00 (in 3.418331 seconds) chunkcleanupworker stdout | 2023-08-08 06:27:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:28 UTC)" (scheduled at 2023-08-08 06:27:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:27:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:27:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:27:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:27:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:39.417660+00:00 (in 9.999469 seconds) gcworker stdout | 2023-08-08 06:27:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:39 UTC)" (scheduled at 2023-08-08 06:27:29.417660+00:00) gcworker stdout | 2023-08-08 06:27:29,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266449418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:27:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:27:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:27:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:27:29,801 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:27:29,958 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:27:30,985 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:27:31,882 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:27:32,017 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:27:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:27:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:42.293757+00:00 (in 9.999469 seconds) chunkcleanupworker stdout | 2023-08-08 06:27:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:42 UTC)" (scheduled at 2023-08-08 06:27:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:27:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:27:32,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 32, 294763), True, datetime.datetime(2023, 8, 8, 6, 27, 32, 294763), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:27:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:27:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:27:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:42 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:27:32,442 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:27:32,875 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:27:33,186 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:27:33,601 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:27:33,662 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:27:33,668 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:27:33,671 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:27:33,674 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:27:33,679 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:27:33,689 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:27:33,694 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:27:33,698 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:27:33,705 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:27:34,828 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:27:34,837 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:27:34,837 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:27:34,841 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:27:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:45.371682+00:00 (in 9.999400 seconds) notificationworker stdout | 2023-08-08 06:27:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:45 UTC)" (scheduled at 2023-08-08 06:27:35.371682+00:00) notificationworker stdout | 2023-08-08 06:27:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:27:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 35, 372663), True, datetime.datetime(2023, 8, 8, 6, 27, 35, 372663), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:27:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:27:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:27:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:27:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:54.393627+00:00 (in 17.813262 seconds) namespacegcworker stdout | 2023-08-08 06:27:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:36 UTC)" (scheduled at 2023-08-08 06:27:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:27:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:27:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:27:37,180 [200] [DEBUG] [app] Starting request: urn:request:932d1ed3-3a56-4aca-9685-a23b1b83eda3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:27:37,180 [199] [DEBUG] [app] Starting request: urn:request:5d1b4395-a0df-421b-9abb-5c8f7c9ca18a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:27:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:37,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:37,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:27:37,185 [210] [DEBUG] [app] Starting request: urn:request:0c037afd-b5e9-47b5-9b68-f47180b98224 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:27:37,186 [210] [DEBUG] [app] Ending request: urn:request:0c037afd-b5e9-47b5-9b68-f47180b98224 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0c037afd-b5e9-47b5-9b68-f47180b98224', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:27:37,186 [207] [DEBUG] [app] Starting request: urn:request:ae39f8a4-b2b8-47ea-ad9e-3354032f7b76 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:27:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:27:37,186 [207] [DEBUG] [app] Ending request: urn:request:ae39f8a4-b2b8-47ea-ad9e-3354032f7b76 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ae39f8a4-b2b8-47ea-ad9e-3354032f7b76', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:27:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:27:37,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:27:37,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:37,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:37,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:37,192 [197] [DEBUG] [app] Starting request: urn:request:e0a607cd-3d10-492b-aac7-a3f6ad2130f5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:27:37,192 [200] [DEBUG] [app] Starting request: urn:request:419e3544-0cea-4da4-8f5a-bc26c60ed9b4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:27:37,192 [200] [DEBUG] [app] Ending request: urn:request:419e3544-0cea-4da4-8f5a-bc26c60ed9b4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:419e3544-0cea-4da4-8f5a-bc26c60ed9b4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:27:37,192 [197] [DEBUG] [app] Ending request: urn:request:e0a607cd-3d10-492b-aac7-a3f6ad2130f5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e0a607cd-3d10-492b-aac7-a3f6ad2130f5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:27:37,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:27:37,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:27:37,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:37,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:37,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:27:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:27:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:27:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:07.197412+00:00 (in 29.999495 seconds) buildlogsarchiver stdout | 2023-08-08 06:27:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:07 UTC)" (scheduled at 2023-08-08 06:27:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:27:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 27, 37, 198387), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:27:37,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:27:37,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:27:37,204 [199] [DEBUG] [app] Ending request: urn:request:5d1b4395-a0df-421b-9abb-5c8f7c9ca18a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5d1b4395-a0df-421b-9abb-5c8f7c9ca18a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:27:37,205 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:27:37,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:27:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:27:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:27:37,205 [200] [DEBUG] [app] Ending request: urn:request:932d1ed3-3a56-4aca-9685-a23b1b83eda3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:932d1ed3-3a56-4aca-9685-a23b1b83eda3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:27:37,205 [200] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:27:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:27:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) gunicorn-web stdout | 2023-08-08 06:27:37,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:27:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:27:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:27:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:27:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:27:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:27:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:27:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:49.417660+00:00 (in 9.999458 seconds) gcworker stdout | 2023-08-08 06:27:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:49 UTC)" (scheduled at 2023-08-08 06:27:39.417660+00:00) gcworker stdout | 2023-08-08 06:27:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266459418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:27:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:27:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:27:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:27:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:27:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:52.293757+00:00 (in 9.999481 seconds) chunkcleanupworker stdout | 2023-08-08 06:27:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:52 UTC)" (scheduled at 2023-08-08 06:27:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:27:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:27:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 42, 294682), True, datetime.datetime(2023, 8, 8, 6, 27, 42, 294682), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:27:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:27:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:27:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:52 UTC)" executed successfully builder stdout | 2023-08-08 06:27:44,852 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:27:44,861 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:27:44,861 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:27:44,865 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:27:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:55.371682+00:00 (in 9.999504 seconds) notificationworker stdout | 2023-08-08 06:27:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:55 UTC)" (scheduled at 2023-08-08 06:27:45.371682+00:00) notificationworker stdout | 2023-08-08 06:27:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:27:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 45, 372538), True, datetime.datetime(2023, 8, 8, 6, 27, 45, 372538), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:27:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:27:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:27:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:27:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:25.514341+00:00 (in 38.115279 seconds) exportactionlogsworker stdout | 2023-08-08 06:27:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:47 UTC)" (scheduled at 2023-08-08 06:27:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:27:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:27:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 47, 399553), True, datetime.datetime(2023, 8, 8, 6, 27, 47, 399553), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:27:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:27:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:27:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:27:49,048 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:27:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:27:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:59.417660+00:00 (in 9.999245 seconds) gcworker stdout | 2023-08-08 06:27:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:59 UTC)" (scheduled at 2023-08-08 06:27:49.417660+00:00) gcworker stdout | 2023-08-08 06:27:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266469418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:27:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:27:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:27:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:27:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:27:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:27:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:27:56.890741+00:00 (in 7.202904 seconds) securityworker stdout | 2023-08-08 06:27:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:19 UTC)" (scheduled at 2023-08-08 06:27:49.687363+00:00) securityworker stdout | 2023-08-08 06:27:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:27:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:27:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:27:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:27:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:27:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:27:50,233 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:27:51,314 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:27:51,786 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:27:52,179 [197] [DEBUG] [app] Starting request: urn:request:f03a46ed-6633-4f83-a32e-1d8e3bb26c08 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:27:52,179 [199] [DEBUG] [app] Starting request: urn:request:98388b51-5c4a-470b-b959-85b1082b6ef2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:27:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:27:52,184 [210] [DEBUG] [app] Starting request: urn:request:2be54e4e-dc9f-42fb-91e6-017a8d43c525 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:27:52,184 [206] [DEBUG] [app] Starting request: urn:request:3b1d53b8-e6ef-4497-813e-2c2ef4c99b19 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:27:52,185 [210] [DEBUG] [app] Ending request: urn:request:2be54e4e-dc9f-42fb-91e6-017a8d43c525 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2be54e4e-dc9f-42fb-91e6-017a8d43c525', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:27:52,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:27:52,185 [206] [DEBUG] [app] Ending request: urn:request:3b1d53b8-e6ef-4497-813e-2c2ef4c99b19 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3b1d53b8-e6ef-4497-813e-2c2ef4c99b19', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:27:52,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:27:52,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:27:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:52,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:27:52,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:52,190 [197] [DEBUG] [app] Starting request: urn:request:ce145e13-44a6-4032-b953-8609ce39b4a7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:27:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:27:52,190 [197] [DEBUG] [app] Ending request: urn:request:ce145e13-44a6-4032-b953-8609ce39b4a7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ce145e13-44a6-4032-b953-8609ce39b4a7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:27:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:27:52,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:52,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:27:52,191 [200] [DEBUG] [app] Starting request: urn:request:9f26fb28-4ee0-4806-89be-f8ce218b4a6d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:27:52,192 [200] [DEBUG] [app] Ending request: urn:request:9f26fb28-4ee0-4806-89be-f8ce218b4a6d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9f26fb28-4ee0-4806-89be-f8ce218b4a6d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:27:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:27:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:27:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:27:52,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:27:52,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:27:52,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:27:52,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:27:52,202 [197] [DEBUG] [app] Ending request: urn:request:f03a46ed-6633-4f83-a32e-1d8e3bb26c08 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f03a46ed-6633-4f83-a32e-1d8e3bb26c08', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:27:52,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:27:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:27:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:27:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:27:52,203 [199] [DEBUG] [app] Ending request: urn:request:98388b51-5c4a-470b-b959-85b1082b6ef2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:98388b51-5c4a-470b-b959-85b1082b6ef2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:27:52,203 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:27:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:27:52,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:27:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:27:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:27:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:02.293757+00:00 (in 9.999451 seconds) chunkcleanupworker stdout | 2023-08-08 06:27:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:02 UTC)" (scheduled at 2023-08-08 06:27:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:27:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:27:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 52, 294718), True, datetime.datetime(2023, 8, 8, 6, 27, 52, 294718), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:27:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:27:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:27:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:27:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:27:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:13.097375+00:00 (in 18.812725 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:27:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:54 UTC)" (scheduled at 2023-08-08 06:27:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:27:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:27:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:27:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:27:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:36.579841+00:00 (in 42.185701 seconds) namespacegcworker stdout | 2023-08-08 06:27:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:54 UTC)" (scheduled at 2023-08-08 06:27:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:27:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:27:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 54, 394718), True, datetime.datetime(2023, 8, 8, 6, 27, 54, 394718), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:27:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:27:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:27:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:28:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:27:54,405 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:27:54,875 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:27:54,883 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:27:54,883 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:27:54,888 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:27:55,353 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:27:55,357 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:27:55,360 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:27:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:27:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:02.178530+00:00 (in 6.806363 seconds) notificationworker stdout | 2023-08-08 06:27:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:05 UTC)" (scheduled at 2023-08-08 06:27:55.371682+00:00) notificationworker stdout | 2023-08-08 06:27:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:27:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 27, 55, 372520), True, datetime.datetime(2023, 8, 8, 6, 27, 55, 372520), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:27:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:27:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:27:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:27:56,291 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:27:56,688 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:27:56,719 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:27:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:27:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:19.687363+00:00 (in 22.796136 seconds) securityworker stdout | 2023-08-08 06:27:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:26 UTC)" (scheduled at 2023-08-08 06:27:56.890741+00:00) securityworker stdout | 2023-08-08 06:27:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:27:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:27:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:27:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:27:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:27:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:27:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:27:56,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:27:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:27:56,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:27:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 22, 56, 896051), 1, 2]) securityworker stdout | 2023-08-08 06:27:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 22, 56, 896051), 1, 2]) securityworker stdout | 2023-08-08 06:27:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:27:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:27:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:27:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:27:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:27:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:27:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:27:56,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:27:56,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:27:56,908 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:26 UTC)" executed successfully teamsyncworker stdout | 2023-08-08 06:27:57,741 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:27:57,747 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:27:57,954 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:27:58,045 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:27:58,050 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:27:58,060 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:27:58,313 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:27:58,507 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:27:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:27:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:09.417660+00:00 (in 9.999444 seconds) gcworker stdout | 2023-08-08 06:27:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:09 UTC)" (scheduled at 2023-08-08 06:27:59.417660+00:00) gcworker stdout | 2023-08-08 06:27:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266479418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:27:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:27:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:27:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:27:59,819 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:27:59,972 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:28:01,005 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:28:01,919 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:28:02,045 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} notificationworker stdout | 2023-08-08 06:28:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:28:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:05.371682+00:00 (in 3.192627 seconds) notificationworker stdout | 2023-08-08 06:28:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:02 UTC)" (scheduled at 2023-08-08 06:28:02.178530+00:00) notificationworker stdout | 2023-08-08 06:28:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:28:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:28:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:28:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:12.293757+00:00 (in 9.999458 seconds) chunkcleanupworker stdout | 2023-08-08 06:28:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:12 UTC)" (scheduled at 2023-08-08 06:28:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:28:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:28:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 2, 294684), True, datetime.datetime(2023, 8, 8, 6, 28, 2, 294684), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:28:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:28:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:28:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:12 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:28:02,451 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:28:02,906 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:28:03,213 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:28:03,640 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:28:03,674 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:28:03,680 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:28:03,683 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:28:03,686 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:28:03,691 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:28:03,697 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:28:03,704 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:28:03,707 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:28:03,715 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:28:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:28:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:09.793658+00:00 (in 5.605703 seconds) repositorygcworker stdout | 2023-08-08 06:28:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:04 UTC)" (scheduled at 2023-08-08 06:28:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:28:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:28:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:04 UTC)" executed successfully builder stdout | 2023-08-08 06:28:04,891 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:28:04,900 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:28:04,900 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:28:04,904 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:28:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:15.371682+00:00 (in 9.999466 seconds) notificationworker stdout | 2023-08-08 06:28:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:15 UTC)" (scheduled at 2023-08-08 06:28:05.371682+00:00) notificationworker stdout | 2023-08-08 06:28:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:28:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 5, 372687), True, datetime.datetime(2023, 8, 8, 6, 28, 5, 372687), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:28:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:28:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:28:07,180 [197] [DEBUG] [app] Starting request: urn:request:3f9c64bf-3828-4554-88ee-2d5571c8019a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:28:07,180 [200] [DEBUG] [app] Starting request: urn:request:a600e140-ac48-4950-8caf-abff5e9f137f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:28:07,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:07,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:07,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:28:07,186 [210] [DEBUG] [app] Starting request: urn:request:5a274841-c7db-4479-9506-85d08eb699e4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:28:07,186 [210] [DEBUG] [app] Ending request: urn:request:5a274841-c7db-4479-9506-85d08eb699e4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5a274841-c7db-4479-9506-85d08eb699e4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:28:07,186 [206] [DEBUG] [app] Starting request: urn:request:f3ab7bfc-216c-4f2e-80fd-217f844c4b03 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:28:07,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:28:07,187 [206] [DEBUG] [app] Ending request: urn:request:f3ab7bfc-216c-4f2e-80fd-217f844c4b03 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f3ab7bfc-216c-4f2e-80fd-217f844c4b03', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:28:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:28:07,187 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:28:07,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:07,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:07,191 [200] [DEBUG] [app] Starting request: urn:request:94dbddf8-a34d-44a9-88f6-2fea9a839e7d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:28:07,192 [200] [DEBUG] [app] Ending request: urn:request:94dbddf8-a34d-44a9-88f6-2fea9a839e7d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:94dbddf8-a34d-44a9-88f6-2fea9a839e7d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:28:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:28:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:07,193 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:28:07,194 [199] [DEBUG] [app] Starting request: urn:request:87c0cab6-8830-4196-82f9-bfa7f3c9e1ad (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:28:07,194 [199] [DEBUG] [app] Ending request: urn:request:87c0cab6-8830-4196-82f9-bfa7f3c9e1ad (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:87c0cab6-8830-4196-82f9-bfa7f3c9e1ad', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:28:07,195 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:28:07,195 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:07,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:28:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:28:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:28:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:37.197412+00:00 (in 29.999507 seconds) buildlogsarchiver stdout | 2023-08-08 06:28:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:37 UTC)" (scheduled at 2023-08-08 06:28:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:28:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 28, 7, 198362), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:28:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:28:07,203 [197] [DEBUG] [app] Ending request: urn:request:3f9c64bf-3828-4554-88ee-2d5571c8019a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3f9c64bf-3828-4554-88ee-2d5571c8019a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:28:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:28:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:28:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:28:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:28:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:28:07,205 [200] [DEBUG] [app] Ending request: urn:request:a600e140-ac48-4950-8caf-abff5e9f137f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a600e140-ac48-4950-8caf-abff5e9f137f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:28:07,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:28:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:28:07,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:28:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:28:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:28:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:28:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:28:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:28:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:28:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:19.417660+00:00 (in 9.999465 seconds) gcworker stdout | 2023-08-08 06:28:09,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:19 UTC)" (scheduled at 2023-08-08 06:28:09.417660+00:00) gcworker stdout | 2023-08-08 06:28:09,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266489419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:28:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:28:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:28:09,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:28:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:28:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:04.187429+00:00 (in 54.393206 seconds) repositorygcworker stdout | 2023-08-08 06:28:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:09 UTC)" (scheduled at 2023-08-08 06:28:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:28:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:28:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 9, 795330), True, datetime.datetime(2023, 8, 8, 6, 28, 9, 795330), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:28:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:28:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:28:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:28:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:28:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:22.293757+00:00 (in 9.999420 seconds) chunkcleanupworker stdout | 2023-08-08 06:28:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:22 UTC)" (scheduled at 2023-08-08 06:28:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:28:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:28:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 12, 294723), True, datetime.datetime(2023, 8, 8, 6, 28, 12, 294723), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:28:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:28:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:28:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:28:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:28:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:54.284093+00:00 (in 41.186164 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:28:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:13 UTC)" (scheduled at 2023-08-08 06:28:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:28:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:28:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 13, 98408), True, datetime.datetime(2023, 8, 8, 6, 28, 13, 98408), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:28:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:28:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:28:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:13 UTC)" executed successfully builder stdout | 2023-08-08 06:28:14,914 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:28:14,924 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:28:14,924 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:28:14,929 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:28:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:25.371682+00:00 (in 9.999467 seconds) notificationworker stdout | 2023-08-08 06:28:15,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:25 UTC)" (scheduled at 2023-08-08 06:28:15.371682+00:00) notificationworker stdout | 2023-08-08 06:28:15,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:28:15,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 15, 373471), True, datetime.datetime(2023, 8, 8, 6, 28, 15, 373471), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:28:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:28:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:28:19,080 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:28:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:28:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:29.417660+00:00 (in 9.999420 seconds) gcworker stdout | 2023-08-08 06:28:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:29 UTC)" (scheduled at 2023-08-08 06:28:19.417660+00:00) gcworker stdout | 2023-08-08 06:28:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266499418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:28:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:28:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:28:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:28:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:28:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:26.890741+00:00 (in 7.202812 seconds) securityworker stdout | 2023-08-08 06:28:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:49 UTC)" (scheduled at 2023-08-08 06:28:19.687363+00:00) securityworker stdout | 2023-08-08 06:28:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:28:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:28:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:28:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:28:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:28:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:28:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 23, 19, 692377), 1, 2]) securityworker stdout | 2023-08-08 06:28:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 23, 19, 692377), 1, 2]) securityworker stdout | 2023-08-08 06:28:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:28:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:28:20,242 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:28:21,355 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:28:21,807 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:28:22,178 [197] [DEBUG] [app] Starting request: urn:request:6091934f-0128-4fac-938c-886856076ab8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:28:22,178 [200] [DEBUG] [app] Starting request: urn:request:89f733fc-45c2-43ca-9374-d060f2094ec2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:28:22,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:22,182 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:28:22,184 [203] [DEBUG] [app] Starting request: urn:request:55435a41-0160-40aa-99f5-e72433864cd1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:28:22,184 [203] [DEBUG] [app] Ending request: urn:request:55435a41-0160-40aa-99f5-e72433864cd1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:55435a41-0160-40aa-99f5-e72433864cd1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:28:22,184 [205] [DEBUG] [app] Starting request: urn:request:c7c41db8-f942-447c-8397-6db1bb94e2c9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:28:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:28:22,185 [205] [DEBUG] [app] Ending request: urn:request:c7c41db8-f942-447c-8397-6db1bb94e2c9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c7c41db8-f942-447c-8397-6db1bb94e2c9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:28:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:28:22,185 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:28:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:22,186 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:22,190 [198] [DEBUG] [app] Starting request: urn:request:a567af80-dc84-46cc-bc84-8e86c47e49e3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:28:22,191 [198] [DEBUG] [app] Ending request: urn:request:a567af80-dc84-46cc-bc84-8e86c47e49e3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a567af80-dc84-46cc-bc84-8e86c47e49e3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:28:22,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:28:22,191 [199] [DEBUG] [app] Starting request: urn:request:ce817de5-69c7-4d7f-bc00-592ace95ef71 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:28:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:22,192 [199] [DEBUG] [app] Ending request: urn:request:ce817de5-69c7-4d7f-bc00-592ace95ef71 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ce817de5-69c7-4d7f-bc00-592ace95ef71', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:28:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:28:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:28:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:22,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:28:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:28:22,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:28:22,203 [197] [DEBUG] [app] Ending request: urn:request:6091934f-0128-4fac-938c-886856076ab8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6091934f-0128-4fac-938c-886856076ab8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:28:22,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:28:22,204 [200] [DEBUG] [app] Ending request: urn:request:89f733fc-45c2-43ca-9374-d060f2094ec2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:89f733fc-45c2-43ca-9374-d060f2094ec2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:28:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:28:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:28:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:28:22,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:28:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:28:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:28:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:28:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:28:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:28.874906+00:00 (in 6.580562 seconds) chunkcleanupworker stdout | 2023-08-08 06:28:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:32 UTC)" (scheduled at 2023-08-08 06:28:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:28:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:28:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 22, 294752), True, datetime.datetime(2023, 8, 8, 6, 28, 22, 294752), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:28:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:28:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:28:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:28:24,436 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:28:24,933 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:28:24,941 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:28:24,941 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:28:24,946 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:28:25,362 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:28:25,365 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:28:25,368 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:28:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:28:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:35.371682+00:00 (in 9.999497 seconds) notificationworker stdout | 2023-08-08 06:28:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:35 UTC)" (scheduled at 2023-08-08 06:28:25.371682+00:00) notificationworker stdout | 2023-08-08 06:28:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:28:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 25, 372770), True, datetime.datetime(2023, 8, 8, 6, 28, 25, 372770), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:28:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:28:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:28:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:28:25,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:33.514094+00:00 (in 7.999261 seconds) exportactionlogsworker stdout | 2023-08-08 06:28:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:25 UTC)" (scheduled at 2023-08-08 06:28:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:28:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:28:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:28:26,329 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:28:26,705 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:28:26,737 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:28:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:28:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:49.687363+00:00 (in 22.796066 seconds) securityworker stdout | 2023-08-08 06:28:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:56 UTC)" (scheduled at 2023-08-08 06:28:26.890741+00:00) securityworker stdout | 2023-08-08 06:28:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:28:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:28:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:28:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:28:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:28:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:28:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:28:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:28:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:28:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:28:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 23, 26, 896684), 1, 2]) securityworker stdout | 2023-08-08 06:28:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 23, 26, 896684), 1, 2]) securityworker stdout | 2023-08-08 06:28:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:28:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:28:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:28:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:28:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:28:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:28:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:28:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:28:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:28:27,756 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:28:27,778 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:28:27,966 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:28:28,060 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:28:28,063 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:28:28,073 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:28:28,329 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:28:28,543 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:28:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:28:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:32.293757+00:00 (in 3.418291 seconds) chunkcleanupworker stdout | 2023-08-08 06:28:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:28 UTC)" (scheduled at 2023-08-08 06:28:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:28:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:28:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:28:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:28:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:39.417660+00:00 (in 9.999446 seconds) gcworker stdout | 2023-08-08 06:28:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:39 UTC)" (scheduled at 2023-08-08 06:28:29.417660+00:00) gcworker stdout | 2023-08-08 06:28:29,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266509418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:28:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:28:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:28:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:28:29,855 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:28:30,012 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:28:31,023 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:28:31,937 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:28:32,065 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:28:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:28:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:42.293757+00:00 (in 9.999418 seconds) chunkcleanupworker stdout | 2023-08-08 06:28:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:42 UTC)" (scheduled at 2023-08-08 06:28:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:28:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:28:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 32, 294851), True, datetime.datetime(2023, 8, 8, 6, 28, 32, 294851), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:28:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:28:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:28:32,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:42 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:28:32,460 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:28:32,937 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:28:33,251 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} exportactionlogsworker stdout | 2023-08-08 06:28:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:28:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:47.398551+00:00 (in 13.883891 seconds) exportactionlogsworker stdout | 2023-08-08 06:28:33,514 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:33:33 UTC)" (scheduled at 2023-08-08 06:28:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:28:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 28, 33, 515096), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:28:33,523 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 28, 33, 515096), True, datetime.datetime(2023, 8, 8, 6, 28, 33, 515096), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:28:33,524 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 28, 33, 515096), True, datetime.datetime(2023, 8, 8, 6, 28, 33, 515096), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 28, 33, 515096), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:28:33,526 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:28:33,526 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:33:33 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:28:33,675 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:28:33,686 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:28:33,690 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:28:33,693 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:28:33,699 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:28:33,702 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:28:33,705 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:28:33,714 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:28:33,717 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:28:33,724 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:28:34,956 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:28:34,966 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:28:34,966 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:28:34,970 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:28:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:45.371682+00:00 (in 9.999411 seconds) notificationworker stdout | 2023-08-08 06:28:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:45 UTC)" (scheduled at 2023-08-08 06:28:35.371682+00:00) notificationworker stdout | 2023-08-08 06:28:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:28:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 35, 372841), True, datetime.datetime(2023, 8, 8, 6, 28, 35, 372841), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:28:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:28:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:28:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:28:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:54.393627+00:00 (in 17.813198 seconds) namespacegcworker stdout | 2023-08-08 06:28:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:36 UTC)" (scheduled at 2023-08-08 06:28:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:28:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:28:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:28:37,179 [198] [DEBUG] [app] Starting request: urn:request:34b01a44-90cc-4ffb-bccd-200da3a85ec6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:28:37,179 [197] [DEBUG] [app] Starting request: urn:request:b8f7f518-bb24-4e29-a17c-769cda13698e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:28:37,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:37,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:28:37,185 [208] [DEBUG] [app] Starting request: urn:request:e221386a-6719-4c1d-b2b0-44113f4f5ebc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:28:37,185 [203] [DEBUG] [app] Starting request: urn:request:c0459616-11cb-4fd6-a7b8-ddae994c8c90 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:28:37,186 [203] [DEBUG] [app] Ending request: urn:request:c0459616-11cb-4fd6-a7b8-ddae994c8c90 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c0459616-11cb-4fd6-a7b8-ddae994c8c90', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:28:37,186 [208] [DEBUG] [app] Ending request: urn:request:e221386a-6719-4c1d-b2b0-44113f4f5ebc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e221386a-6719-4c1d-b2b0-44113f4f5ebc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:28:37,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:28:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:28:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:37,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:37,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:37,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:37,192 [199] [DEBUG] [app] Starting request: urn:request:d790421f-f70f-4914-a04f-57b57aa0111f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:28:37,192 [198] [DEBUG] [app] Starting request: urn:request:0b4e1ecb-49f6-4031-b494-be043c744180 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:28:37,192 [199] [DEBUG] [app] Ending request: urn:request:d790421f-f70f-4914-a04f-57b57aa0111f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d790421f-f70f-4914-a04f-57b57aa0111f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:28:37,192 [198] [DEBUG] [app] Ending request: urn:request:0b4e1ecb-49f6-4031-b494-be043c744180 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0b4e1ecb-49f6-4031-b494-be043c744180', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:28:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:28:37,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:28:37,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:37,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:28:37,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:28:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:28:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:07.197412+00:00 (in 29.999481 seconds) buildlogsarchiver stdout | 2023-08-08 06:28:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:07 UTC)" (scheduled at 2023-08-08 06:28:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:28:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 28, 37, 198343), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:28:37,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:28:37,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:28:37,205 [198] [DEBUG] [app] Ending request: urn:request:34b01a44-90cc-4ffb-bccd-200da3a85ec6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:34b01a44-90cc-4ffb-bccd-200da3a85ec6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:28:37,205 [197] [DEBUG] [app] Ending request: urn:request:b8f7f518-bb24-4e29-a17c-769cda13698e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b8f7f518-bb24-4e29-a17c-769cda13698e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:28:37,205 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:28:37,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:28:37,205 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:28:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:28:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:28:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:28:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:28:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) buildlogsarchiver stdout | 2023-08-08 06:28:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:28:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:28:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:28:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:28:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:28:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:28:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:49.417660+00:00 (in 9.999474 seconds) gcworker stdout | 2023-08-08 06:28:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:49 UTC)" (scheduled at 2023-08-08 06:28:39.417660+00:00) gcworker stdout | 2023-08-08 06:28:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266519418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:28:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:28:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:28:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:28:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:28:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:52.293757+00:00 (in 9.999452 seconds) chunkcleanupworker stdout | 2023-08-08 06:28:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:52 UTC)" (scheduled at 2023-08-08 06:28:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:28:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:28:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 42, 294798), True, datetime.datetime(2023, 8, 8, 6, 28, 42, 294798), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:28:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:28:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:28:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:52 UTC)" executed successfully builder stdout | 2023-08-08 06:28:44,979 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:28:44,988 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:28:44,988 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:28:44,993 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:28:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:49.178248+00:00 (in 3.805996 seconds) notificationworker stdout | 2023-08-08 06:28:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:55 UTC)" (scheduled at 2023-08-08 06:28:45.371682+00:00) notificationworker stdout | 2023-08-08 06:28:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:28:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 45, 372883), True, datetime.datetime(2023, 8, 8, 6, 28, 45, 372883), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:28:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:28:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:28:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:28:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:25.514341+00:00 (in 38.115297 seconds) exportactionlogsworker stdout | 2023-08-08 06:28:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:47 UTC)" (scheduled at 2023-08-08 06:28:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:28:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:28:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 47, 399563), True, datetime.datetime(2023, 8, 8, 6, 28, 47, 399563), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:28:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:28:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:28:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:28:49,119 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} notificationworker stdout | 2023-08-08 06:28:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:28:49,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:55.371682+00:00 (in 6.192790 seconds) notificationworker stdout | 2023-08-08 06:28:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:33:49 UTC)" (scheduled at 2023-08-08 06:28:49.178248+00:00) notificationworker stdout | 2023-08-08 06:28:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 28, 49, 179427), 'notification/%']) notificationworker stdout | 2023-08-08 06:28:49,187 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 28, 49, 179427), True, datetime.datetime(2023, 8, 8, 6, 28, 49, 179427), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:28:49,189 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 28, 49, 179427), True, datetime.datetime(2023, 8, 8, 6, 28, 49, 179427), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 28, 49, 179427), 'notification/%']) notificationworker stdout | 2023-08-08 06:28:49,191 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:49,191 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:33:49 UTC)" executed successfully gcworker stdout | 2023-08-08 06:28:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:28:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:59.417660+00:00 (in 9.999433 seconds) gcworker stdout | 2023-08-08 06:28:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:59 UTC)" (scheduled at 2023-08-08 06:28:49.417660+00:00) gcworker stdout | 2023-08-08 06:28:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266529418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:28:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:28:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:28:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:28:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:28:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:28:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:28:56.890741+00:00 (in 7.202875 seconds) securityworker stdout | 2023-08-08 06:28:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:19 UTC)" (scheduled at 2023-08-08 06:28:49.687363+00:00) securityworker stdout | 2023-08-08 06:28:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:28:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:28:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:28:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:28:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:28:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:28:50,277 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:28:51,392 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:28:51,844 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:28:52,178 [199] [DEBUG] [app] Starting request: urn:request:f2dcf43c-0ae3-4e54-b5c0-175d7b7c8e22 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:28:52,179 [197] [DEBUG] [app] Starting request: urn:request:53450039-dc4b-407d-8355-213a75ecfe6c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:28:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:52,182 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:28:52,183 [208] [DEBUG] [app] Starting request: urn:request:b353bbe8-c389-4356-be16-a2af61635c73 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:28:52,184 [208] [DEBUG] [app] Ending request: urn:request:b353bbe8-c389-4356-be16-a2af61635c73 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b353bbe8-c389-4356-be16-a2af61635c73', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:28:52,184 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:28:52,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:28:52,185 [210] [DEBUG] [app] Starting request: urn:request:e5b9380b-4ada-4e0e-be7e-cbc93c0d45f9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:28:52,185 [210] [DEBUG] [app] Ending request: urn:request:e5b9380b-4ada-4e0e-be7e-cbc93c0d45f9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e5b9380b-4ada-4e0e-be7e-cbc93c0d45f9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:28:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:28:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:52,186 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:28:52,188 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:28:52,190 [199] [DEBUG] [app] Starting request: urn:request:c566b25d-4a2c-4185-a002-94cbda4b23f3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:28:52,190 [199] [DEBUG] [app] Ending request: urn:request:c566b25d-4a2c-4185-a002-94cbda4b23f3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c566b25d-4a2c-4185-a002-94cbda4b23f3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:28:52,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:28:52,191 [198] [DEBUG] [app] Starting request: urn:request:dd22f67b-2588-491b-98ce-246e556eb0a5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:28:52,191 [198] [DEBUG] [app] Ending request: urn:request:dd22f67b-2588-491b-98ce-246e556eb0a5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:dd22f67b-2588-491b-98ce-246e556eb0a5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:28:52,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:28:52,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:28:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:28:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:28:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:28:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:28:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:28:52,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:28:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:28:52,203 [197] [DEBUG] [app] Ending request: urn:request:53450039-dc4b-407d-8355-213a75ecfe6c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:53450039-dc4b-407d-8355-213a75ecfe6c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:28:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:28:52,203 [199] [DEBUG] [app] Ending request: urn:request:f2dcf43c-0ae3-4e54-b5c0-175d7b7c8e22 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f2dcf43c-0ae3-4e54-b5c0-175d7b7c8e22', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:28:52,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:28:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:28:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:28:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:28:52,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:28:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:28:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:28:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:28:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:02.293757+00:00 (in 9.999413 seconds) chunkcleanupworker stdout | 2023-08-08 06:28:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:02 UTC)" (scheduled at 2023-08-08 06:28:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:28:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:28:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 52, 295000), True, datetime.datetime(2023, 8, 8, 6, 28, 52, 295000), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:28:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:28:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:28:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:28:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:28:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:13.097375+00:00 (in 18.812730 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:28:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:54 UTC)" (scheduled at 2023-08-08 06:28:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:28:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:28:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:28:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:28:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:36.579841+00:00 (in 42.185531 seconds) namespacegcworker stdout | 2023-08-08 06:28:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:54 UTC)" (scheduled at 2023-08-08 06:28:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:28:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:28:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 54, 394876), True, datetime.datetime(2023, 8, 8, 6, 28, 54, 394876), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:28:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:28:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:28:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:29:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:28:54,459 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:28:54,999 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:28:55,008 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:28:55,008 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:28:55,012 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:28:55,372 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} notificationworker stdout | 2023-08-08 06:28:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:28:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:02.178530+00:00 (in 6.806313 seconds) notificationworker stdout | 2023-08-08 06:28:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:05 UTC)" (scheduled at 2023-08-08 06:28:55.371682+00:00) notificationworker stdout | 2023-08-08 06:28:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:28:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 28, 55, 372786), True, datetime.datetime(2023, 8, 8, 6, 28, 55, 372786), 0, 'notification/%', 50, 1, 0]) gunicorn-secscan stdout | 2023-08-08 06:28:55,374 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:28:55,377 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} notificationworker stdout | 2023-08-08 06:28:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:28:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:28:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:05 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:28:56,354 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:28:56,739 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:28:56,775 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:28:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:28:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:19.687363+00:00 (in 22.796099 seconds) securityworker stdout | 2023-08-08 06:28:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:26 UTC)" (scheduled at 2023-08-08 06:28:56.890741+00:00) securityworker stdout | 2023-08-08 06:28:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:28:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:28:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:28:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:28:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:28:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:28:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:28:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:28:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:28:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:28:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 23, 56, 896346), 1, 2]) securityworker stdout | 2023-08-08 06:28:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 23, 56, 896346), 1, 2]) securityworker stdout | 2023-08-08 06:28:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:28:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:28:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:28:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:28:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:28:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:28:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:28:56,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:28:56,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:28:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:28:57,764 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:28:57,815 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:28:57,978 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:28:58,076 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:28:58,079 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:28:58,085 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:28:58,368 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:28:58,558 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:28:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:28:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:09.417660+00:00 (in 9.999487 seconds) gcworker stdout | 2023-08-08 06:28:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:09 UTC)" (scheduled at 2023-08-08 06:28:59.417660+00:00) gcworker stdout | 2023-08-08 06:28:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266539418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:28:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:28:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:28:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:28:59,873 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:29:00,022 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:29:01,060 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:29:01,949 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:29:02,102 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} notificationworker stdout | 2023-08-08 06:29:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:29:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:05.371682+00:00 (in 3.192668 seconds) notificationworker stdout | 2023-08-08 06:29:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:02 UTC)" (scheduled at 2023-08-08 06:29:02.178530+00:00) notificationworker stdout | 2023-08-08 06:29:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:29:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:29:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:29:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:12.293757+00:00 (in 9.999329 seconds) chunkcleanupworker stdout | 2023-08-08 06:29:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:12 UTC)" (scheduled at 2023-08-08 06:29:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:29:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:29:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 2, 294964), True, datetime.datetime(2023, 8, 8, 6, 29, 2, 294964), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:29:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:29:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:29:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:12 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:29:02,497 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:29:02,974 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:29:03,277 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:29:03,697 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:29:03,700 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:29:03,704 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:29:03,709 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} expiredappspecifictokenworker stdout | 2023-08-08 06:29:03,712 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:29:03,714 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:29:03,717 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:29:03,726 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:29:03,729 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:29:03,732 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:29:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:29:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:09.793658+00:00 (in 5.605732 seconds) repositorygcworker stdout | 2023-08-08 06:29:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:04 UTC)" (scheduled at 2023-08-08 06:29:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:29:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:29:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:04 UTC)" executed successfully builder stdout | 2023-08-08 06:29:05,023 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:29:05,032 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:29:05,032 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:29:05,036 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:29:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:15.371682+00:00 (in 9.999499 seconds) notificationworker stdout | 2023-08-08 06:29:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:15 UTC)" (scheduled at 2023-08-08 06:29:05.371682+00:00) notificationworker stdout | 2023-08-08 06:29:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:29:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 5, 372666), True, datetime.datetime(2023, 8, 8, 6, 29, 5, 372666), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:29:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:29:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:29:07,179 [197] [DEBUG] [app] Starting request: urn:request:d8a2f8eb-0ae6-44ca-b21c-a2a3dc7fe0c8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:29:07,179 [200] [DEBUG] [app] Starting request: urn:request:d45be15e-7732-4149-9737-87b5e18a20c2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:29:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:29:07,184 [203] [DEBUG] [app] Starting request: urn:request:b4ada720-8788-4234-8bce-daf3448aa2c3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:29:07,184 [210] [DEBUG] [app] Starting request: urn:request:dc598afe-9322-4176-9d6f-891f59a0f011 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:29:07,185 [203] [DEBUG] [app] Ending request: urn:request:b4ada720-8788-4234-8bce-daf3448aa2c3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b4ada720-8788-4234-8bce-daf3448aa2c3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:29:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:29:07,185 [210] [DEBUG] [app] Ending request: urn:request:dc598afe-9322-4176-9d6f-891f59a0f011 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:dc598afe-9322-4176-9d6f-891f59a0f011', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:29:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:29:07,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:29:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:07,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:07,190 [198] [DEBUG] [app] Starting request: urn:request:38c0d8eb-0b75-46da-9c0f-50c43826689a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:29:07,190 [198] [DEBUG] [app] Ending request: urn:request:38c0d8eb-0b75-46da-9c0f-50c43826689a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:38c0d8eb-0b75-46da-9c0f-50c43826689a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:29:07,190 [199] [DEBUG] [app] Starting request: urn:request:d4a2daa9-9eca-4973-b9ad-b864de44cced (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:29:07,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:29:07,191 [199] [DEBUG] [app] Ending request: urn:request:d4a2daa9-9eca-4973-b9ad-b864de44cced (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d4a2daa9-9eca-4973-b9ad-b864de44cced', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:29:07,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:29:07,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:29:07,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:07,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:29:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:29:07,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:29:07,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:29:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:29:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:37.197412+00:00 (in 29.999552 seconds) buildlogsarchiver stdout | 2023-08-08 06:29:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:37 UTC)" (scheduled at 2023-08-08 06:29:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:29:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 29, 7, 198692), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:29:07,202 [197] [DEBUG] [app] Ending request: urn:request:d8a2f8eb-0ae6-44ca-b21c-a2a3dc7fe0c8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d8a2f8eb-0ae6-44ca-b21c-a2a3dc7fe0c8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:29:07,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:29:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:29:07,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:29:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:29:07,203 [200] [DEBUG] [app] Ending request: urn:request:d45be15e-7732-4149-9737-87b5e18a20c2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d45be15e-7732-4149-9737-87b5e18a20c2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:29:07,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:29:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:29:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:29:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:29:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:29:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:29:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:29:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:29:07,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:29:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:29:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:19.417660+00:00 (in 9.999471 seconds) gcworker stdout | 2023-08-08 06:29:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:19 UTC)" (scheduled at 2023-08-08 06:29:09.417660+00:00) gcworker stdout | 2023-08-08 06:29:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266549418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:29:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:29:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:29:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:29:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:29:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:19.187230+00:00 (in 9.393038 seconds) repositorygcworker stdout | 2023-08-08 06:29:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:09 UTC)" (scheduled at 2023-08-08 06:29:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:29:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:29:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 9, 794689), True, datetime.datetime(2023, 8, 8, 6, 29, 9, 794689), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:29:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:29:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:29:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:29:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:29:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:22.293757+00:00 (in 9.999414 seconds) chunkcleanupworker stdout | 2023-08-08 06:29:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:22 UTC)" (scheduled at 2023-08-08 06:29:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:29:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:29:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 12, 294824), True, datetime.datetime(2023, 8, 8, 6, 29, 12, 294824), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:29:12,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:29:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:29:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:29:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:29:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:54.284093+00:00 (in 41.185966 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:29:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:13 UTC)" (scheduled at 2023-08-08 06:29:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:29:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:29:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 13, 98535), True, datetime.datetime(2023, 8, 8, 6, 29, 13, 98535), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:29:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:29:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:29:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:13 UTC)" executed successfully builder stdout | 2023-08-08 06:29:15,042 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:29:15,051 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:29:15,051 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:29:15,056 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:29:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:25.371682+00:00 (in 9.999453 seconds) notificationworker stdout | 2023-08-08 06:29:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:25 UTC)" (scheduled at 2023-08-08 06:29:15.371682+00:00) notificationworker stdout | 2023-08-08 06:29:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:29:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 15, 372916), True, datetime.datetime(2023, 8, 8, 6, 29, 15, 372916), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:29:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:29:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:25 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:29:15,662 [200] [DEBUG] [app] Starting request: urn:request:dca41b9f-9102-4019-a90d-f5bea81b39c7 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:29:15,662 [197] [DEBUG] [app] Starting request: urn:request:0c7327ad-bb5b-4173-bb79-4762972ad015 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:29:15,662 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:29:15,662 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:29:15,662 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:29:15,662 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:29:15,663 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:29:15,663 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:29:15,670 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:29:15,670 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:29:15,670 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:29:15,670 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:29:15,670 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:29:15,670 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:29:15,670 [197] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:29:15,670 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:29:15,670 [200] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:29:15,670 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:29:15,670 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:29:15,670 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:29:15,670 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:29:15,670 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:29:15,670 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:29:15,670 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:29:15,671 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 41, 15, 670746), datetime.datetime(2023, 8, 8, 6, 29, 15, 670797)]) gunicorn-web stdout | 2023-08-08 06:29:15,672 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:29:15,672 [197] [DEBUG] [app] Ending request: urn:request:0c7327ad-bb5b-4173-bb79-4762972ad015 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:0c7327ad-bb5b-4173-bb79-4762972ad015', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:29:15,673 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:29:15 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 62 0.013) gunicorn-web stdout | 2023-08-08 06:29:15,673 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:29:15 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:29:15,676 [200] [DEBUG] [app] Ending request: urn:request:dca41b9f-9102-4019-a90d-f5bea81b39c7 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:dca41b9f-9102-4019-a90d-f5bea81b39c7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:29:15,677 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:29:15 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 829 0.017) gunicorn-web stdout | 2023-08-08 06:29:15,677 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:29:15 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:29:19,157 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} repositorygcworker stdout | 2023-08-08 06:29:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:29:19,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:04.187429+00:00 (in 44.999599 seconds) repositorygcworker stdout | 2023-08-08 06:29:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:34:19 UTC)" (scheduled at 2023-08-08 06:29:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:29:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 29, 19, 188365), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:29:19,196 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 29, 19, 188365), True, datetime.datetime(2023, 8, 8, 6, 29, 19, 188365), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:29:19,198 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 29, 19, 188365), True, datetime.datetime(2023, 8, 8, 6, 29, 19, 188365), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 29, 19, 188365), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:29:19,199 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:29:19,199 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:34:19 UTC)" executed successfully gcworker stdout | 2023-08-08 06:29:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:29:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:29.417660+00:00 (in 9.999451 seconds) gcworker stdout | 2023-08-08 06:29:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:29 UTC)" (scheduled at 2023-08-08 06:29:19.417660+00:00) gcworker stdout | 2023-08-08 06:29:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266559418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:29:19,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:29:19,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:29:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:29:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:29:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:26.890741+00:00 (in 7.202910 seconds) securityworker stdout | 2023-08-08 06:29:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:49 UTC)" (scheduled at 2023-08-08 06:29:19.687363+00:00) securityworker stdout | 2023-08-08 06:29:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:29:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:29:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:29:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:29:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:29:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:29:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 24, 19, 691804), 1, 2]) securityworker stdout | 2023-08-08 06:29:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 24, 19, 691804), 1, 2]) securityworker stdout | 2023-08-08 06:29:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:29:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:29:20,298 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:29:21,402 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:29:21,858 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:29:22,179 [198] [DEBUG] [app] Starting request: urn:request:b8bc072f-04cb-49ff-a0c4-a55284bbb111 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:29:22,180 [200] [DEBUG] [app] Starting request: urn:request:0627c488-5d21-442b-ac41-7f63c848bcf6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:29:22,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:22,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:22,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:22,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:29:22,185 [210] [DEBUG] [app] Starting request: urn:request:c6ff07e9-83f0-41f0-a6e6-38303914ad6f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:29:22,185 [210] [DEBUG] [app] Ending request: urn:request:c6ff07e9-83f0-41f0-a6e6-38303914ad6f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c6ff07e9-83f0-41f0-a6e6-38303914ad6f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:29:22,186 [207] [DEBUG] [app] Starting request: urn:request:ca2758ce-a758-41ff-bf1f-86894551624b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:29:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:29:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:29:22,186 [207] [DEBUG] [app] Ending request: urn:request:ca2758ce-a758-41ff-bf1f-86894551624b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ca2758ce-a758-41ff-bf1f-86894551624b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:29:22,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:29:22,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:22,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:22,191 [199] [DEBUG] [app] Starting request: urn:request:6bebb280-d87f-441c-a581-62ad4014f938 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:29:22,191 [199] [DEBUG] [app] Ending request: urn:request:6bebb280-d87f-441c-a581-62ad4014f938 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6bebb280-d87f-441c-a581-62ad4014f938', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:29:22,191 [197] [DEBUG] [app] Starting request: urn:request:b63e3ec5-04cd-4c1c-8ba7-5c1ad5def2b6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:29:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:29:22,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:22,192 [197] [DEBUG] [app] Ending request: urn:request:b63e3ec5-04cd-4c1c-8ba7-5c1ad5def2b6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b63e3ec5-04cd-4c1c-8ba7-5c1ad5def2b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:29:22,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:29:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:29:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:22,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:29:22,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:29:22,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:29:22,203 [198] [DEBUG] [app] Ending request: urn:request:b8bc072f-04cb-49ff-a0c4-a55284bbb111 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b8bc072f-04cb-49ff-a0c4-a55284bbb111', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:29:22,203 [200] [DEBUG] [app] Ending request: urn:request:0627c488-5d21-442b-ac41-7f63c848bcf6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0627c488-5d21-442b-ac41-7f63c848bcf6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:29:22,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:29:22,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:29:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:29:22,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:29:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:29:22,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:29:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:29:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.023) chunkcleanupworker stdout | 2023-08-08 06:29:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:29:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:28.874906+00:00 (in 6.580466 seconds) chunkcleanupworker stdout | 2023-08-08 06:29:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:32 UTC)" (scheduled at 2023-08-08 06:29:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:29:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:29:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 22, 295007), True, datetime.datetime(2023, 8, 8, 6, 29, 22, 295007), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:29:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:29:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:29:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:29:24,496 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:29:25,066 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:29:25,076 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:29:25,076 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:29:25,081 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:29:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:35.371682+00:00 (in 9.999379 seconds) notificationworker stdout | 2023-08-08 06:29:25,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:35 UTC)" (scheduled at 2023-08-08 06:29:25.371682+00:00) notificationworker stdout | 2023-08-08 06:29:25,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:29:25,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 25, 373331), True, datetime.datetime(2023, 8, 8, 6, 29, 25, 373331), 0, 'notification/%', 50, 1, 0]) gunicorn-secscan stdout | 2023-08-08 06:29:25,380 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} notificationworker stdout | 2023-08-08 06:29:25,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:29:25,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:25,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:35 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:29:25,383 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:29:25,386 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} exportactionlogsworker stdout | 2023-08-08 06:29:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:29:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:47.398551+00:00 (in 21.883662 seconds) exportactionlogsworker stdout | 2023-08-08 06:29:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:25 UTC)" (scheduled at 2023-08-08 06:29:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:29:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:29:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:29:26,366 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:29:26,777 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:29:26,812 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:29:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:29:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:49.687363+00:00 (in 22.796026 seconds) securityworker stdout | 2023-08-08 06:29:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:56 UTC)" (scheduled at 2023-08-08 06:29:26.890741+00:00) securityworker stdout | 2023-08-08 06:29:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:29:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:29:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:29:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:29:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:29:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:29:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:29:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:29:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:29:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:29:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 24, 26, 896607), 1, 2]) securityworker stdout | 2023-08-08 06:29:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 24, 26, 896607), 1, 2]) securityworker stdout | 2023-08-08 06:29:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:26,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:29:26,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:29:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:29:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:29:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:29:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:29:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:29:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:29:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:29:27,774 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:29:27,835 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:29:27,991 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:29:28,092 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:29:28,095 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:29:28,098 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:29:28,381 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:29:28,584 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:29:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:29:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:32.293757+00:00 (in 3.418300 seconds) chunkcleanupworker stdout | 2023-08-08 06:29:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:28 UTC)" (scheduled at 2023-08-08 06:29:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:29:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:29:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:29:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:29:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:39.417660+00:00 (in 9.999479 seconds) gcworker stdout | 2023-08-08 06:29:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:39 UTC)" (scheduled at 2023-08-08 06:29:29.417660+00:00) gcworker stdout | 2023-08-08 06:29:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266569418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:29:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:29:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:29:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:29:29,910 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:29:30,046 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:29:31,096 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:29:31,988 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:29:32,132 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:29:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:29:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:42.293757+00:00 (in 9.999479 seconds) chunkcleanupworker stdout | 2023-08-08 06:29:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:42 UTC)" (scheduled at 2023-08-08 06:29:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:29:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:29:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 32, 294742), True, datetime.datetime(2023, 8, 8, 6, 29, 32, 294742), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:29:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:29:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:29:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:42 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:29:32,513 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:29:32,996 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:29:33,289 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:29:33,706 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:29:33,710 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:29:33,713 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:29:33,718 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:29:33,726 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:29:33,729 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} expiredappspecifictokenworker stdout | 2023-08-08 06:29:33,732 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-registry stdout | 2023-08-08 06:29:33,735 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:29:33,738 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:29:33,741 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} builder stdout | 2023-08-08 06:29:35,091 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:29:35,100 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:29:35,100 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:29:35,104 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:29:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:45.371682+00:00 (in 9.999478 seconds) notificationworker stdout | 2023-08-08 06:29:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:45 UTC)" (scheduled at 2023-08-08 06:29:35.371682+00:00) notificationworker stdout | 2023-08-08 06:29:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:29:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 35, 372586), True, datetime.datetime(2023, 8, 8, 6, 29, 35, 372586), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:29:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:29:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:29:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:29:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:39.579576+00:00 (in 2.999222 seconds) namespacegcworker stdout | 2023-08-08 06:29:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:36 UTC)" (scheduled at 2023-08-08 06:29:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:29:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:29:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:29:37,179 [197] [DEBUG] [app] Starting request: urn:request:13961b99-534c-4e8f-94b7-9f2e14e921f2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:29:37,179 [198] [DEBUG] [app] Starting request: urn:request:556f0b8b-6f8a-48c5-b16e-5ff5e516ddcc (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:29:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:37,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:37,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:29:37,185 [203] [DEBUG] [app] Starting request: urn:request:322ba932-4e62-4db3-becc-fab55782e5a0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:29:37,185 [206] [DEBUG] [app] Starting request: urn:request:0a732212-217a-467c-8294-e13c1297420b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:29:37,185 [203] [DEBUG] [app] Ending request: urn:request:322ba932-4e62-4db3-becc-fab55782e5a0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:322ba932-4e62-4db3-becc-fab55782e5a0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:29:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:29:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:29:37,187 [206] [DEBUG] [app] Ending request: urn:request:0a732212-217a-467c-8294-e13c1297420b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0a732212-217a-467c-8294-e13c1297420b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:29:37,187 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-web stdout | 2023-08-08 06:29:37,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:37,189 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:37,191 [198] [DEBUG] [app] Starting request: urn:request:542ae8f9-6a2c-4f40-8941-24c2e2f6b91a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:29:37,191 [198] [DEBUG] [app] Ending request: urn:request:542ae8f9-6a2c-4f40-8941-24c2e2f6b91a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:542ae8f9-6a2c-4f40-8941-24c2e2f6b91a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:29:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:29:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:37,192 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:29:37,193 [199] [DEBUG] [app] Starting request: urn:request:05c47104-973a-4bdc-8287-60226865ed5e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:29:37,194 [199] [DEBUG] [app] Ending request: urn:request:05c47104-973a-4bdc-8287-60226865ed5e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:05c47104-973a-4bdc-8287-60226865ed5e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:29:37,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:29:37,194 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:37,195 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:29:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:29:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:29:37,197 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:07.197412+00:00 (in 29.999569 seconds) buildlogsarchiver stdout | 2023-08-08 06:29:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:07 UTC)" (scheduled at 2023-08-08 06:29:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:29:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 29, 37, 198162), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:29:37,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:29:37,202 [197] [DEBUG] [app] Ending request: urn:request:13961b99-534c-4e8f-94b7-9f2e14e921f2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:13961b99-534c-4e8f-94b7-9f2e14e921f2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:29:37,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:29:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:29:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:29:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:29:37,205 [198] [DEBUG] [app] Ending request: urn:request:556f0b8b-6f8a-48c5-b16e-5ff5e516ddcc (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:556f0b8b-6f8a-48c5-b16e-5ff5e516ddcc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 06:29:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:29:37,205 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:29:37,205 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:29:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:29:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:29:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:29:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:29:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:29:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:29:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:29:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:49.417660+00:00 (in 9.999460 seconds) gcworker stdout | 2023-08-08 06:29:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:49 UTC)" (scheduled at 2023-08-08 06:29:39.417660+00:00) gcworker stdout | 2023-08-08 06:29:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266579418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:29:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:29:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:29:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:29:39,579 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:29:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:54.393627+00:00 (in 14.813577 seconds) namespacegcworker stdout | 2023-08-08 06:29:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:34:39 UTC)" (scheduled at 2023-08-08 06:29:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:29:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 29, 39, 580941), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:29:39,588 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 29, 39, 580941), True, datetime.datetime(2023, 8, 8, 6, 29, 39, 580941), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:29:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 29, 39, 580941), True, datetime.datetime(2023, 8, 8, 6, 29, 39, 580941), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 29, 39, 580941), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:29:39,591 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:29:39,591 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:34:39 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:29:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:29:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:46.874674+00:00 (in 4.580426 seconds) chunkcleanupworker stdout | 2023-08-08 06:29:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:52 UTC)" (scheduled at 2023-08-08 06:29:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:29:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:29:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 42, 294802), True, datetime.datetime(2023, 8, 8, 6, 29, 42, 294802), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:29:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:29:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:29:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:52 UTC)" executed successfully builder stdout | 2023-08-08 06:29:45,114 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:29:45,123 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:29:45,123 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:29:45,128 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:29:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:55.371682+00:00 (in 9.999485 seconds) notificationworker stdout | 2023-08-08 06:29:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:55 UTC)" (scheduled at 2023-08-08 06:29:45.371682+00:00) notificationworker stdout | 2023-08-08 06:29:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:29:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 45, 372828), True, datetime.datetime(2023, 8, 8, 6, 29, 45, 372828), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:29:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:29:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:29:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:29:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:52.293757+00:00 (in 5.418586 seconds) chunkcleanupworker stdout | 2023-08-08 06:29:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:34:46 UTC)" (scheduled at 2023-08-08 06:29:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:29:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 29, 46, 875534), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:29:46,883 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 29, 46, 875534), True, datetime.datetime(2023, 8, 8, 6, 29, 46, 875534), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:29:46,885 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 29, 46, 875534), True, datetime.datetime(2023, 8, 8, 6, 29, 46, 875534), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 29, 46, 875534), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:29:46,886 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:29:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:34:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:29:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:29:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:25.514341+00:00 (in 38.115301 seconds) exportactionlogsworker stdout | 2023-08-08 06:29:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:47 UTC)" (scheduled at 2023-08-08 06:29:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:29:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:29:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 47, 399400), True, datetime.datetime(2023, 8, 8, 6, 29, 47, 399400), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:29:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:29:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:29:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:29:49,194 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:29:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:29:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:59.417660+00:00 (in 9.999319 seconds) gcworker stdout | 2023-08-08 06:29:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:59 UTC)" (scheduled at 2023-08-08 06:29:49.417660+00:00) gcworker stdout | 2023-08-08 06:29:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266589418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:29:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:29:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:29:49,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:29:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:29:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:29:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:29:56.890741+00:00 (in 7.202843 seconds) securityworker stdout | 2023-08-08 06:29:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:19 UTC)" (scheduled at 2023-08-08 06:29:49.687363+00:00) securityworker stdout | 2023-08-08 06:29:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:29:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:29:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:29:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:29:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:29:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:29:50,329 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:29:51,439 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:29:51,869 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:29:52,178 [197] [DEBUG] [app] Starting request: urn:request:64f6bcc6-8fd9-4e8b-a1cc-34bdb5dccd0d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:29:52,178 [198] [DEBUG] [app] Starting request: urn:request:fa93db75-296d-47eb-89dc-3ba51beb4fb7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:29:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:52,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:29:52,184 [210] [DEBUG] [app] Starting request: urn:request:b55c0d3e-1be2-4cfb-9f90-0663899533b2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:29:52,184 [207] [DEBUG] [app] Starting request: urn:request:7f1d0073-b389-4d2c-8918-4b6dce348682 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:29:52,185 [210] [DEBUG] [app] Ending request: urn:request:b55c0d3e-1be2-4cfb-9f90-0663899533b2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b55c0d3e-1be2-4cfb-9f90-0663899533b2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:29:52,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:29:52,185 [207] [DEBUG] [app] Ending request: urn:request:7f1d0073-b389-4d2c-8918-4b6dce348682 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7f1d0073-b389-4d2c-8918-4b6dce348682', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:29:52,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:29:52,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:29:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:29:52,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:52,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:29:52,190 [197] [DEBUG] [app] Starting request: urn:request:314b579c-93e9-4500-be13-cd4c77bf32ac (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:29:52,191 [197] [DEBUG] [app] Ending request: urn:request:314b579c-93e9-4500-be13-cd4c77bf32ac (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:314b579c-93e9-4500-be13-cd4c77bf32ac', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:29:52,191 [200] [DEBUG] [app] Starting request: urn:request:036d075c-88f9-45e1-b133-4847e262ad8b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:29:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:29:52,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:52,192 [200] [DEBUG] [app] Ending request: urn:request:036d075c-88f9-45e1-b133-4847e262ad8b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:036d075c-88f9-45e1-b133-4847e262ad8b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:29:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:29:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:29:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:29:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:29:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:29:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:29:52,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:29:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:29:52,202 [198] [DEBUG] [app] Ending request: urn:request:fa93db75-296d-47eb-89dc-3ba51beb4fb7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fa93db75-296d-47eb-89dc-3ba51beb4fb7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:29:52,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:29:52,202 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:29:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:29:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:29:52,203 [197] [DEBUG] [app] Ending request: urn:request:64f6bcc6-8fd9-4e8b-a1cc-34bdb5dccd0d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:64f6bcc6-8fd9-4e8b-a1cc-34bdb5dccd0d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:29:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:29:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:29:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:29:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:29:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:29:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:02.293757+00:00 (in 9.999510 seconds) chunkcleanupworker stdout | 2023-08-08 06:29:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:02 UTC)" (scheduled at 2023-08-08 06:29:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:29:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:29:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 52, 294583), True, datetime.datetime(2023, 8, 8, 6, 29, 52, 294583), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:29:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:29:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:29:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:29:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:29:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:13.097375+00:00 (in 18.812684 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:29:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:54 UTC)" (scheduled at 2023-08-08 06:29:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:29:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:29:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:29:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:29:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:36.579841+00:00 (in 42.185701 seconds) namespacegcworker stdout | 2023-08-08 06:29:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:54 UTC)" (scheduled at 2023-08-08 06:29:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:29:54,395 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:29:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 54, 395038), True, datetime.datetime(2023, 8, 8, 6, 29, 54, 395038), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:29:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:29:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:29:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:30:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:29:54,517 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:29:55,138 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:29:55,148 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:29:55,148 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:29:55,152 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:29:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:02.178530+00:00 (in 6.806253 seconds) notificationworker stdout | 2023-08-08 06:29:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:05 UTC)" (scheduled at 2023-08-08 06:29:55.371682+00:00) notificationworker stdout | 2023-08-08 06:29:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:29:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 29, 55, 372656), True, datetime.datetime(2023, 8, 8, 6, 29, 55, 372656), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:29:55,379 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:29:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:29:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:29:55,389 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:29:55,392 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:29:55,395 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:29:56,403 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:29:56,785 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:29:56,849 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:29:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:29:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:19.687363+00:00 (in 22.796111 seconds) securityworker stdout | 2023-08-08 06:29:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:26 UTC)" (scheduled at 2023-08-08 06:29:56.890741+00:00) securityworker stdout | 2023-08-08 06:29:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:29:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:29:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:29:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:29:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:29:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:29:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:29:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:29:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:29:56,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:29:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 24, 56, 896314), 1, 2]) securityworker stdout | 2023-08-08 06:29:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 24, 56, 896314), 1, 2]) securityworker stdout | 2023-08-08 06:29:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:29:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:29:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:29:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:29:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:29:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:29:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:29:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:29:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:29:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:29:57,783 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:29:57,865 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:29:58,004 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:29:58,107 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:29:58,110 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:29:58,112 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:29:58,418 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:29:58,621 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:29:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:29:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:09.417660+00:00 (in 9.999461 seconds) gcworker stdout | 2023-08-08 06:29:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:09 UTC)" (scheduled at 2023-08-08 06:29:59.417660+00:00) gcworker stdout | 2023-08-08 06:29:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266599418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:29:59,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:29:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:29:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:29:59,948 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:30:00,055 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:30:01,109 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:30:02,026 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:30:02,170 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} notificationworker stdout | 2023-08-08 06:30:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:30:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:05.371682+00:00 (in 3.192659 seconds) notificationworker stdout | 2023-08-08 06:30:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:02 UTC)" (scheduled at 2023-08-08 06:30:02.178530+00:00) notificationworker stdout | 2023-08-08 06:30:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:30:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:30:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:30:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:12.293757+00:00 (in 9.999459 seconds) chunkcleanupworker stdout | 2023-08-08 06:30:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:12 UTC)" (scheduled at 2023-08-08 06:30:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:30:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:30:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 2, 294722), True, datetime.datetime(2023, 8, 8, 6, 30, 2, 294722), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:30:02,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:30:02,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:30:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:12 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:30:02,550 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:30:03,014 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:30:03,327 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:30:03,718 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:30:03,722 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:30:03,725 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:30:03,727 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:30:03,737 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:30:03,741 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:30:03,744 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:30:03,747 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:30:03,751 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:30:03,770 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} repositorygcworker stdout | 2023-08-08 06:30:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:30:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:09.793658+00:00 (in 5.605732 seconds) repositorygcworker stdout | 2023-08-08 06:30:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:04 UTC)" (scheduled at 2023-08-08 06:30:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:30:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:30:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:04 UTC)" executed successfully builder stdout | 2023-08-08 06:30:05,163 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:30:05,172 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:30:05,172 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:30:05,177 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:30:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:15.371682+00:00 (in 9.999352 seconds) notificationworker stdout | 2023-08-08 06:30:05,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:15 UTC)" (scheduled at 2023-08-08 06:30:05.371682+00:00) notificationworker stdout | 2023-08-08 06:30:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:30:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 5, 373346), True, datetime.datetime(2023, 8, 8, 6, 30, 5, 373346), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:30:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:30:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:30:07,179 [199] [DEBUG] [app] Starting request: urn:request:e75e9bfc-865b-4253-8e01-f8fb80a9c0e7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:30:07,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:07,181 [200] [DEBUG] [app] Starting request: urn:request:fdf5eb55-bdf2-460e-9eb3-3116201ba536 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:30:07,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:30:07,184 [208] [DEBUG] [app] Starting request: urn:request:c2b8236e-ebad-4f0e-bd75-beda63c43c34 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:30:07,185 [208] [DEBUG] [app] Ending request: urn:request:c2b8236e-ebad-4f0e-bd75-beda63c43c34 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c2b8236e-ebad-4f0e-bd75-beda63c43c34', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:30:07,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:30:07,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:07,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:30:07,186 [208] [DEBUG] [app] Starting request: urn:request:e527297f-ebac-495e-b3d8-f04cc1f1774a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:30:07,186 [208] [DEBUG] [app] Ending request: urn:request:e527297f-ebac-495e-b3d8-f04cc1f1774a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e527297f-ebac-495e-b3d8-f04cc1f1774a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:30:07,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.001) gunicorn-web stdout | 2023-08-08 06:30:07,186 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:07,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:07,190 [199] [DEBUG] [app] Starting request: urn:request:6dc2edbb-6fd9-4c0a-a506-1172077904fa (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:30:07,191 [199] [DEBUG] [app] Ending request: urn:request:6dc2edbb-6fd9-4c0a-a506-1172077904fa (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6dc2edbb-6fd9-4c0a-a506-1172077904fa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:30:07,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:30:07,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:07,191 [197] [DEBUG] [app] Starting request: urn:request:de232caf-f0ab-463c-a607-3904488e37ef (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:30:07,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:30:07,192 [197] [DEBUG] [app] Ending request: urn:request:de232caf-f0ab-463c-a607-3904488e37ef (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:de232caf-f0ab-463c-a607-3904488e37ef', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:30:07,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:30:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:30:07,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:30:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:30:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:30:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:37.197412+00:00 (in 29.999470 seconds) buildlogsarchiver stdout | 2023-08-08 06:30:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:37 UTC)" (scheduled at 2023-08-08 06:30:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:30:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 30, 7, 198390), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:30:07,203 [199] [DEBUG] [app] Ending request: urn:request:e75e9bfc-865b-4253-8e01-f8fb80a9c0e7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e75e9bfc-865b-4253-8e01-f8fb80a9c0e7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:30:07,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:30:07,203 [200] [DEBUG] [app] Ending request: urn:request:fdf5eb55-bdf2-460e-9eb3-3116201ba536 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fdf5eb55-bdf2-460e-9eb3-3116201ba536', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:30:07,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:30:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:30:07,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:30:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:30:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:30:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:30:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.023 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:30:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:30:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:30:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:30:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:30:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:30:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:30:09,419 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:19.417660+00:00 (in 9.999017 seconds) gcworker stdout | 2023-08-08 06:30:09,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:19 UTC)" (scheduled at 2023-08-08 06:30:09.417660+00:00) gcworker stdout | 2023-08-08 06:30:09,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266609419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:30:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:30:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:30:09,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:30:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:30:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:04.187429+00:00 (in 54.392749 seconds) repositorygcworker stdout | 2023-08-08 06:30:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:09 UTC)" (scheduled at 2023-08-08 06:30:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:30:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:30:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 9, 795132), True, datetime.datetime(2023, 8, 8, 6, 30, 9, 795132), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:30:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:30:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:30:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:30:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:30:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:22.293757+00:00 (in 9.999430 seconds) chunkcleanupworker stdout | 2023-08-08 06:30:12,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:22 UTC)" (scheduled at 2023-08-08 06:30:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:30:12,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:30:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 12, 295327), True, datetime.datetime(2023, 8, 8, 6, 30, 12, 295327), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:30:12,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:30:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:30:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:30:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:30:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:54.284093+00:00 (in 41.186218 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:30:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:13 UTC)" (scheduled at 2023-08-08 06:30:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:30:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:30:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 13, 98248), True, datetime.datetime(2023, 8, 8, 6, 30, 13, 98248), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:30:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:30:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:30:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:13 UTC)" executed successfully builder stdout | 2023-08-08 06:30:15,187 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:30:15,197 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:30:15,197 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:30:15,202 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:30:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:25.371682+00:00 (in 9.999328 seconds) notificationworker stdout | 2023-08-08 06:30:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:25 UTC)" (scheduled at 2023-08-08 06:30:15.371682+00:00) notificationworker stdout | 2023-08-08 06:30:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:30:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 15, 372847), True, datetime.datetime(2023, 8, 8, 6, 30, 15, 372847), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:30:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:30:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:30:19,216 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:30:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:30:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:29.417660+00:00 (in 9.999012 seconds) gcworker stdout | 2023-08-08 06:30:19,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:29 UTC)" (scheduled at 2023-08-08 06:30:19.417660+00:00) gcworker stdout | 2023-08-08 06:30:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266619419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:30:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:30:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:30:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:30:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:30:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:26.890741+00:00 (in 7.202898 seconds) securityworker stdout | 2023-08-08 06:30:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:49 UTC)" (scheduled at 2023-08-08 06:30:19.687363+00:00) securityworker stdout | 2023-08-08 06:30:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:30:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:30:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:30:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:30:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:30:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:30:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 25, 19, 692075), 1, 2]) securityworker stdout | 2023-08-08 06:30:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:19,705 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 25, 19, 692075), 1, 2]) securityworker stdout | 2023-08-08 06:30:19,706 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:30:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:30:20,361 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:30:21,477 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:30:21,907 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:30:22,178 [198] [DEBUG] [app] Starting request: urn:request:b6de04b1-227b-4b43-bac7-ceb857646535 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:30:22,178 [197] [DEBUG] [app] Starting request: urn:request:dfcc1c3d-c37d-4bbf-9b21-0b59dce08684 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:30:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:22,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:22,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:30:22,184 [205] [DEBUG] [app] Starting request: urn:request:2793154a-deee-420a-a487-41f86312628d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:30:22,185 [205] [DEBUG] [app] Ending request: urn:request:2793154a-deee-420a-a487-41f86312628d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2793154a-deee-420a-a487-41f86312628d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:30:22,185 [210] [DEBUG] [app] Starting request: urn:request:88a72f47-493d-46da-bd4f-7306173406cf (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:30:22,185 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:30:22,185 [210] [DEBUG] [app] Ending request: urn:request:88a72f47-493d-46da-bd4f-7306173406cf (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:88a72f47-493d-46da-bd4f-7306173406cf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:30:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:30:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:30:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:22,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:22,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:22,190 [200] [DEBUG] [app] Starting request: urn:request:7fe7f006-725a-43a2-b4e5-e3773fe9fb0e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:30:22,191 [197] [DEBUG] [app] Starting request: urn:request:7af88fd1-3a87-4df6-9cf2-fc43b19f1428 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:30:22,191 [197] [DEBUG] [app] Ending request: urn:request:7af88fd1-3a87-4df6-9cf2-fc43b19f1428 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7af88fd1-3a87-4df6-9cf2-fc43b19f1428', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:30:22,191 [200] [DEBUG] [app] Ending request: urn:request:7fe7f006-725a-43a2-b4e5-e3773fe9fb0e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7fe7f006-725a-43a2-b4e5-e3773fe9fb0e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2023-08-08 06:30:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:30:22,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:30:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:22,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:30:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:30:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:30:22,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:30:22,203 [197] [DEBUG] [app] Ending request: urn:request:dfcc1c3d-c37d-4bbf-9b21-0b59dce08684 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:dfcc1c3d-c37d-4bbf-9b21-0b59dce08684', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:30:22,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:30:22,203 [198] [DEBUG] [app] Ending request: urn:request:b6de04b1-227b-4b43-bac7-ceb857646535 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b6de04b1-227b-4b43-bac7-ceb857646535', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:30:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:30:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:30:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:30:22,204 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:30:22,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:30:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:30:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:30:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:30:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:28.874906+00:00 (in 6.580115 seconds) chunkcleanupworker stdout | 2023-08-08 06:30:22,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:32 UTC)" (scheduled at 2023-08-08 06:30:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:30:22,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:30:22,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 22, 295244), True, datetime.datetime(2023, 8, 8, 6, 30, 22, 295244), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:30:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:30:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:30:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:30:24,555 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:30:25,212 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:30:25,222 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:30:25,222 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:30:25,227 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:30:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:35.371682+00:00 (in 9.999471 seconds) notificationworker stdout | 2023-08-08 06:30:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:35 UTC)" (scheduled at 2023-08-08 06:30:25.371682+00:00) notificationworker stdout | 2023-08-08 06:30:25,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:30:25,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 25, 373069), True, datetime.datetime(2023, 8, 8, 6, 30, 25, 373069), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:30:25,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:30:25,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:25,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:35 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:30:25,398 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:30:25,400 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:30:25,403 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} exportactionlogsworker stdout | 2023-08-08 06:30:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:30:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:47.398551+00:00 (in 21.883679 seconds) exportactionlogsworker stdout | 2023-08-08 06:30:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:25 UTC)" (scheduled at 2023-08-08 06:30:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:30:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:30:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:30:26,441 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:30:26,818 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:30:26,857 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:30:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:30:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:49.687363+00:00 (in 22.796146 seconds) securityworker stdout | 2023-08-08 06:30:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:56 UTC)" (scheduled at 2023-08-08 06:30:26.890741+00:00) securityworker stdout | 2023-08-08 06:30:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:30:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:30:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:30:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:30:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:30:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:30:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:30:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:30:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:30:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:30:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 25, 26, 896486), 1, 2]) securityworker stdout | 2023-08-08 06:30:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 25, 26, 896486), 1, 2]) securityworker stdout | 2023-08-08 06:30:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:30:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:30:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:30:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:30:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:30:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:30:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:30:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:30:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:30:27,792 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:30:27,901 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:30:28,018 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:30:28,123 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:30:28,126 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:30:28,129 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:30:28,455 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:30:28,658 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:30:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:30:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:32.293757+00:00 (in 3.418343 seconds) chunkcleanupworker stdout | 2023-08-08 06:30:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:28 UTC)" (scheduled at 2023-08-08 06:30:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:30:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:30:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:30:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:30:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:39.417660+00:00 (in 9.999500 seconds) gcworker stdout | 2023-08-08 06:30:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:39 UTC)" (scheduled at 2023-08-08 06:30:29.417660+00:00) gcworker stdout | 2023-08-08 06:30:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266629418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:30:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:30:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:30:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:30:29,980 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:30:30,087 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:30:31,120 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:30:32,058 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:30:32,177 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:30:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:30:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:42.293757+00:00 (in 9.999502 seconds) chunkcleanupworker stdout | 2023-08-08 06:30:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:42 UTC)" (scheduled at 2023-08-08 06:30:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:30:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:30:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 32, 294695), True, datetime.datetime(2023, 8, 8, 6, 30, 32, 294695), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:30:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:30:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:30:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:42 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:30:32,587 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:30:33,049 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:30:33,353 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:30:33,728 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:30:33,733 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:30:33,738 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:30:33,743 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:30:33,748 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:30:33,755 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:30:33,761 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:30:33,766 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:30:33,770 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:30:33,797 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:30:35,237 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:30:35,247 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:30:35,247 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:30:35,251 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:30:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:45.371682+00:00 (in 9.999387 seconds) notificationworker stdout | 2023-08-08 06:30:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:45 UTC)" (scheduled at 2023-08-08 06:30:35.371682+00:00) notificationworker stdout | 2023-08-08 06:30:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:30:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 35, 372927), True, datetime.datetime(2023, 8, 8, 6, 30, 35, 372927), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:30:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:30:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:30:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:30:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:54.393627+00:00 (in 17.813280 seconds) namespacegcworker stdout | 2023-08-08 06:30:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:36 UTC)" (scheduled at 2023-08-08 06:30:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:30:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:30:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:30:37,180 [197] [DEBUG] [app] Starting request: urn:request:de825aab-a517-4e92-9d1a-3aa2f09e461d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:30:37,180 [200] [DEBUG] [app] Starting request: urn:request:4f2ff507-c048-4458-801d-a87c1d350124 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:30:37,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:30:37,186 [203] [DEBUG] [app] Starting request: urn:request:8dc6b17e-c1c0-4255-b55b-6aaad9b77582 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:30:37,186 [203] [DEBUG] [app] Ending request: urn:request:8dc6b17e-c1c0-4255-b55b-6aaad9b77582 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8dc6b17e-c1c0-4255-b55b-6aaad9b77582', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:30:37,186 [208] [DEBUG] [app] Starting request: urn:request:4bb11e51-e87b-430c-bfe1-fd42de6da8d4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:30:37,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:30:37,187 [208] [DEBUG] [app] Ending request: urn:request:4bb11e51-e87b-430c-bfe1-fd42de6da8d4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4bb11e51-e87b-430c-bfe1-fd42de6da8d4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:30:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:30:37,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:30:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:37,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:37,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:37,192 [197] [DEBUG] [app] Starting request: urn:request:02e392ff-204c-4fe5-a910-f8ad7a3517fd (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:30:37,192 [198] [DEBUG] [app] Starting request: urn:request:4a56ccf6-2800-4e92-89cf-ea428f9627ca (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:30:37,193 [197] [DEBUG] [app] Ending request: urn:request:02e392ff-204c-4fe5-a910-f8ad7a3517fd (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:02e392ff-204c-4fe5-a910-f8ad7a3517fd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:30:37,193 [198] [DEBUG] [app] Ending request: urn:request:4a56ccf6-2800-4e92-89cf-ea428f9627ca (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4a56ccf6-2800-4e92-89cf-ea428f9627ca', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:30:37,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:30:37,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:37,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:30:37,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:37,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:30:37,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:30:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:30:37,197 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:07.197412+00:00 (in 29.999568 seconds) buildlogsarchiver stdout | 2023-08-08 06:30:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:07 UTC)" (scheduled at 2023-08-08 06:30:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:30:37,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:30:37,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:30:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 30, 37, 198256), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:30:37,204 [197] [DEBUG] [app] Ending request: urn:request:de825aab-a517-4e92-9d1a-3aa2f09e461d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:de825aab-a517-4e92-9d1a-3aa2f09e461d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:30:37,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:30:37,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:30:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:30:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:30:37,205 [200] [DEBUG] [app] Ending request: urn:request:4f2ff507-c048-4458-801d-a87c1d350124 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4f2ff507-c048-4458-801d-a87c1d350124', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:30:37,205 [200] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:30:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:30:37,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:30:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:30:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:30:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:30:37,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:30:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:30:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:30:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:30:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:49.417660+00:00 (in 9.998955 seconds) gcworker stdout | 2023-08-08 06:30:39,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:49 UTC)" (scheduled at 2023-08-08 06:30:39.417660+00:00) gcworker stdout | 2023-08-08 06:30:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266639419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:30:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:30:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:30:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:30:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:30:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:52.293757+00:00 (in 9.999508 seconds) chunkcleanupworker stdout | 2023-08-08 06:30:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:52 UTC)" (scheduled at 2023-08-08 06:30:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:30:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:30:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 42, 294718), True, datetime.datetime(2023, 8, 8, 6, 30, 42, 294718), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:30:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:30:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:30:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:52 UTC)" executed successfully builder stdout | 2023-08-08 06:30:45,261 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:30:45,271 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:30:45,271 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:30:45,275 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:30:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:55.371682+00:00 (in 9.999288 seconds) notificationworker stdout | 2023-08-08 06:30:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:55 UTC)" (scheduled at 2023-08-08 06:30:45.371682+00:00) notificationworker stdout | 2023-08-08 06:30:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:30:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 45, 372958), True, datetime.datetime(2023, 8, 8, 6, 30, 45, 372958), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:30:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:30:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:55 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:30:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:30:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:25.514341+00:00 (in 38.115271 seconds) exportactionlogsworker stdout | 2023-08-08 06:30:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:47 UTC)" (scheduled at 2023-08-08 06:30:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:30:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:30:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 47, 399580), True, datetime.datetime(2023, 8, 8, 6, 30, 47, 399580), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:30:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:30:47,409 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:30:47,409 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:30:49,254 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:30:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:30:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:59.417660+00:00 (in 9.999461 seconds) gcworker stdout | 2023-08-08 06:30:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:59 UTC)" (scheduled at 2023-08-08 06:30:49.417660+00:00) gcworker stdout | 2023-08-08 06:30:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266649418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:30:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:30:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:30:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:30:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:30:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:30:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:30:56.890741+00:00 (in 7.202899 seconds) securityworker stdout | 2023-08-08 06:30:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:19 UTC)" (scheduled at 2023-08-08 06:30:49.687363+00:00) securityworker stdout | 2023-08-08 06:30:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:30:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:30:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:30:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:30:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:30:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:30:50,384 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:30:51,514 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:30:51,944 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:30:52,178 [199] [DEBUG] [app] Starting request: urn:request:c08c898c-d63a-4d3e-8068-34b26e90c5a2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:30:52,178 [200] [DEBUG] [app] Starting request: urn:request:a294ade1-faf5-4fc4-bfeb-1abef07349b6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:30:52,179 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:52,182 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:30:52,183 [203] [DEBUG] [app] Starting request: urn:request:dd4ab425-6330-42e5-baf0-fcf47f5eea73 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:30:52,184 [203] [DEBUG] [app] Ending request: urn:request:dd4ab425-6330-42e5-baf0-fcf47f5eea73 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:dd4ab425-6330-42e5-baf0-fcf47f5eea73', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:30:52,184 [209] [DEBUG] [app] Starting request: urn:request:2045de8e-7e57-4a52-9599-7337fa56a4a8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:30:52,184 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:30:52,185 [209] [DEBUG] [app] Ending request: urn:request:2045de8e-7e57-4a52-9599-7337fa56a4a8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2045de8e-7e57-4a52-9599-7337fa56a4a8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:30:52,184 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:30:52,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:30:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:52,186 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:52,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:30:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:30:52,190 [197] [DEBUG] [app] Starting request: urn:request:4b5f8e47-ffb3-45f7-8a52-d88a3ce6abf0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:30:52,191 [198] [DEBUG] [app] Starting request: urn:request:1d5f0f33-8227-491d-80c7-9dfa346bd712 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:30:52,191 [197] [DEBUG] [app] Ending request: urn:request:4b5f8e47-ffb3-45f7-8a52-d88a3ce6abf0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4b5f8e47-ffb3-45f7-8a52-d88a3ce6abf0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:30:52,191 [198] [DEBUG] [app] Ending request: urn:request:1d5f0f33-8227-491d-80c7-9dfa346bd712 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1d5f0f33-8227-491d-80c7-9dfa346bd712', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:30:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:30:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:52,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:30:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:30:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:30:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:30:52,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:30:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:30:52,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:30:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:30:52,203 [199] [DEBUG] [app] Ending request: urn:request:c08c898c-d63a-4d3e-8068-34b26e90c5a2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c08c898c-d63a-4d3e-8068-34b26e90c5a2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:30:52,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:30:52,204 [200] [DEBUG] [app] Ending request: urn:request:a294ade1-faf5-4fc4-bfeb-1abef07349b6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a294ade1-faf5-4fc4-bfeb-1abef07349b6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:30:52,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:30:52,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:30:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:30:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:30:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:30:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:30:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:30:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:30:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:02.293757+00:00 (in 9.999509 seconds) chunkcleanupworker stdout | 2023-08-08 06:30:52,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:02 UTC)" (scheduled at 2023-08-08 06:30:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:30:52,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:30:52,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 52, 295242), True, datetime.datetime(2023, 8, 8, 6, 30, 52, 295242), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:30:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:30:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:30:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:30:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:30:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:13.097375+00:00 (in 18.812720 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:30:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:54 UTC)" (scheduled at 2023-08-08 06:30:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:30:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:30:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:30:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:30:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:36.579841+00:00 (in 42.185717 seconds) namespacegcworker stdout | 2023-08-08 06:30:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:54 UTC)" (scheduled at 2023-08-08 06:30:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:30:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:30:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 54, 394500), True, datetime.datetime(2023, 8, 8, 6, 30, 54, 394500), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:30:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:30:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:30:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:31:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:30:54,593 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:30:55,286 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:30:55,295 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:30:55,295 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:30:55,299 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:30:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:02.178530+00:00 (in 6.806237 seconds) notificationworker stdout | 2023-08-08 06:30:55,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:05 UTC)" (scheduled at 2023-08-08 06:30:55.371682+00:00) notificationworker stdout | 2023-08-08 06:30:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:30:55,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 30, 55, 373422), True, datetime.datetime(2023, 8, 8, 6, 30, 55, 373422), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:30:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:30:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:30:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:30:55,408 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:30:55,411 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:30:55,414 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:30:56,465 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} repositorygcworker stdout | 2023-08-08 06:30:56,856 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:30:56,871 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:30:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:30:56,892 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:19.687363+00:00 (in 22.795511 seconds) securityworker stdout | 2023-08-08 06:30:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:26 UTC)" (scheduled at 2023-08-08 06:30:56.890741+00:00) securityworker stdout | 2023-08-08 06:30:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:30:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:30:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:30:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:30:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:30:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:30:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:30:56,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:30:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:30:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:30:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 25, 56, 897586), 1, 2]) securityworker stdout | 2023-08-08 06:30:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:56,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 25, 56, 897586), 1, 2]) securityworker stdout | 2023-08-08 06:30:56,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:30:56,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:30:56,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:30:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:30:56,911 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:30:56,911 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:30:56,911 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:30:56,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:30:56,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:30:56,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:26 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:30:57,802 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:30:57,940 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:30:58,035 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:30:58,142 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:30:58,146 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:30:58,149 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:30:58,477 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:30:58,667 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:30:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:30:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:09.417660+00:00 (in 9.999472 seconds) gcworker stdout | 2023-08-08 06:30:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:09 UTC)" (scheduled at 2023-08-08 06:30:59.417660+00:00) gcworker stdout | 2023-08-08 06:30:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266659418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:30:59,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:30:59,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:30:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:31:00,010 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:31:00,111 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:31:01,158 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:31:02,071 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} notificationworker stdout | 2023-08-08 06:31:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:31:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:05.371682+00:00 (in 3.192562 seconds) notificationworker stdout | 2023-08-08 06:31:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:02 UTC)" (scheduled at 2023-08-08 06:31:02.178530+00:00) notificationworker stdout | 2023-08-08 06:31:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:31:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:02 UTC)" executed successfully servicekey stdout | 2023-08-08 06:31:02,212 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:31:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:31:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:12.293757+00:00 (in 9.999490 seconds) chunkcleanupworker stdout | 2023-08-08 06:31:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:12 UTC)" (scheduled at 2023-08-08 06:31:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:31:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:31:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 2, 294840), True, datetime.datetime(2023, 8, 8, 6, 31, 2, 294840), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:31:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:31:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:31:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:12 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:31:02,610 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:31:03,082 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:31:03,390 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:31:03,737 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:31:03,741 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:31:03,746 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:31:03,750 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:31:03,756 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:31:03,763 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:31:03,770 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:31:03,775 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:31:03,778 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:31:03,835 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} repositorygcworker stdout | 2023-08-08 06:31:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:31:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:09.793658+00:00 (in 5.605697 seconds) repositorygcworker stdout | 2023-08-08 06:31:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:04 UTC)" (scheduled at 2023-08-08 06:31:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:31:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:31:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:04 UTC)" executed successfully builder stdout | 2023-08-08 06:31:05,304 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:31:05,314 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:31:05,314 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:31:05,319 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:31:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:15.371682+00:00 (in 9.999021 seconds) notificationworker stdout | 2023-08-08 06:31:05,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:15 UTC)" (scheduled at 2023-08-08 06:31:05.371682+00:00) notificationworker stdout | 2023-08-08 06:31:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:31:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 5, 373201), True, datetime.datetime(2023, 8, 8, 6, 31, 5, 373201), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:31:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:31:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:15 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:31:07,179 [197] [DEBUG] [app] Starting request: urn:request:7cf449db-0172-44d2-8de4-8e4fc66210e2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:31:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:07,181 [199] [DEBUG] [app] Starting request: urn:request:cfa6bf38-71b1-483e-83e2-9928a9d7e7d0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:31:07,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:31:07,184 [210] [DEBUG] [app] Starting request: urn:request:c8931d04-1371-4274-b7e5-8b2d3bd3ddb6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:07,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:31:07,185 [210] [DEBUG] [app] Ending request: urn:request:c8931d04-1371-4274-b7e5-8b2d3bd3ddb6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c8931d04-1371-4274-b7e5-8b2d3bd3ddb6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:31:07,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:31:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:31:07,186 [209] [DEBUG] [app] Starting request: urn:request:fb365df3-58f1-4be7-97a5-1026e8808cbb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:31:07,186 [209] [DEBUG] [app] Ending request: urn:request:fb365df3-58f1-4be7-97a5-1026e8808cbb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fb365df3-58f1-4be7-97a5-1026e8808cbb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:31:07,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:31:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:31:07,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:07,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:07,191 [197] [DEBUG] [app] Starting request: urn:request:f3371964-7840-4f47-9307-e003466e3e35 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:07,191 [197] [DEBUG] [app] Ending request: urn:request:f3371964-7840-4f47-9307-e003466e3e35 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f3371964-7840-4f47-9307-e003466e3e35', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:31:07,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:31:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:07,192 [200] [DEBUG] [app] Starting request: urn:request:d73ed2a4-73b8-4b61-bb66-f4728bb017a6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:31:07,192 [200] [DEBUG] [app] Ending request: urn:request:d73ed2a4-73b8-4b61-bb66-f4728bb017a6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d73ed2a4-73b8-4b61-bb66-f4728bb017a6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:31:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:31:07,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:31:07,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:31:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:31:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-web stdout | 2023-08-08 06:31:07,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:31:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:37.197412+00:00 (in 29.999567 seconds) buildlogsarchiver stdout | 2023-08-08 06:31:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:37 UTC)" (scheduled at 2023-08-08 06:31:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:31:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 31, 7, 198342), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:31:07,202 [197] [DEBUG] [app] Ending request: urn:request:7cf449db-0172-44d2-8de4-8e4fc66210e2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7cf449db-0172-44d2-8de4-8e4fc66210e2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:31:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:31:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:31:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:31:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:31:07,204 [199] [DEBUG] [app] Ending request: urn:request:cfa6bf38-71b1-483e-83e2-9928a9d7e7d0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cfa6bf38-71b1-483e-83e2-9928a9d7e7d0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:31:07,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:31:07,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:31:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:31:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:31:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:31:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:31:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:31:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:31:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:31:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:31:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:19.417660+00:00 (in 9.999405 seconds) gcworker stdout | 2023-08-08 06:31:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:19 UTC)" (scheduled at 2023-08-08 06:31:09.417660+00:00) gcworker stdout | 2023-08-08 06:31:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266669418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:31:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:31:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:31:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:31:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:31:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:04.187429+00:00 (in 54.392729 seconds) repositorygcworker stdout | 2023-08-08 06:31:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:09 UTC)" (scheduled at 2023-08-08 06:31:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:31:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:31:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 9, 795167), True, datetime.datetime(2023, 8, 8, 6, 31, 9, 795167), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:31:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:31:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:31:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:31:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:31:12,295 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:22.293757+00:00 (in 9.999022 seconds) chunkcleanupworker stdout | 2023-08-08 06:31:12,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:22 UTC)" (scheduled at 2023-08-08 06:31:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:31:12,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:31:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 12, 295746), True, datetime.datetime(2023, 8, 8, 6, 31, 12, 295746), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:31:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:31:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:31:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:31:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:31:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:31.283850+00:00 (in 18.185945 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:31:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:13 UTC)" (scheduled at 2023-08-08 06:31:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:31:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:31:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 13, 98316), True, datetime.datetime(2023, 8, 8, 6, 31, 13, 98316), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:31:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:31:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:31:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:13 UTC)" executed successfully builder stdout | 2023-08-08 06:31:15,321 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:31:15,329 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:31:15,330 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:31:15,334 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:31:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:25.371682+00:00 (in 9.999421 seconds) notificationworker stdout | 2023-08-08 06:31:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:25 UTC)" (scheduled at 2023-08-08 06:31:15.371682+00:00) notificationworker stdout | 2023-08-08 06:31:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:31:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 15, 372886), True, datetime.datetime(2023, 8, 8, 6, 31, 15, 372886), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:31:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:31:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:25 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:31:19,291 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:31:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:31:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:29.417660+00:00 (in 9.999391 seconds) gcworker stdout | 2023-08-08 06:31:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:29 UTC)" (scheduled at 2023-08-08 06:31:19.417660+00:00) gcworker stdout | 2023-08-08 06:31:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266679418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:31:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:31:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:31:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:31:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:31:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:26.890741+00:00 (in 7.202741 seconds) securityworker stdout | 2023-08-08 06:31:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:49 UTC)" (scheduled at 2023-08-08 06:31:19.687363+00:00) securityworker stdout | 2023-08-08 06:31:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:31:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:31:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:31:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:31:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:31:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:31:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 26, 19, 692138), 1, 2]) securityworker stdout | 2023-08-08 06:31:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 26, 19, 692138), 1, 2]) securityworker stdout | 2023-08-08 06:31:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:31:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:31:20,402 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:31:21,551 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:31:21,961 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:31:22,178 [198] [DEBUG] [app] Starting request: urn:request:a7ba8e06-7799-47d3-90aa-a6e6b2f64c54 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:31:22,179 [197] [DEBUG] [app] Starting request: urn:request:64ab6c30-54c1-4044-9528-620f604bac35 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:31:22,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:22,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:31:22,184 [210] [DEBUG] [app] Starting request: urn:request:ec9dbe1b-ebf0-427a-b2d7-c4f28e9111b6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:31:22,184 [203] [DEBUG] [app] Starting request: urn:request:96db6851-0a8b-4071-b654-bf837a4802d7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:31:22,185 [210] [DEBUG] [app] Ending request: urn:request:ec9dbe1b-ebf0-427a-b2d7-c4f28e9111b6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ec9dbe1b-ebf0-427a-b2d7-c4f28e9111b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:31:22,185 [203] [DEBUG] [app] Ending request: urn:request:96db6851-0a8b-4071-b654-bf837a4802d7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:96db6851-0a8b-4071-b654-bf837a4802d7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:31:22,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:31:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:31:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:22,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:22,191 [198] [DEBUG] [app] Starting request: urn:request:715a285f-004d-40f2-8442-9e078b6c6c17 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:22,192 [198] [DEBUG] [app] Ending request: urn:request:715a285f-004d-40f2-8442-9e078b6c6c17 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:715a285f-004d-40f2-8442-9e078b6c6c17', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:31:22,192 [200] [DEBUG] [app] Starting request: urn:request:692db24b-7eca-4daf-a358-e59377b4fd99 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:31:22,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:22,193 [200] [DEBUG] [app] Ending request: urn:request:692db24b-7eca-4daf-a358-e59377b4fd99 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:692db24b-7eca-4daf-a358-e59377b4fd99', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:31:22,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:31:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:22,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:31:22,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:31:22,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:31:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:31:22,204 [198] [DEBUG] [app] Ending request: urn:request:a7ba8e06-7799-47d3-90aa-a6e6b2f64c54 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a7ba8e06-7799-47d3-90aa-a6e6b2f64c54', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:31:22,204 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:31:22,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:31:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:31:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 06:31:22,204 [197] [DEBUG] [app] Ending request: urn:request:64ab6c30-54c1-4044-9528-620f604bac35 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:64ab6c30-54c1-4044-9528-620f604bac35', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:31:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:31:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:31:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:31:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) chunkcleanupworker stdout | 2023-08-08 06:31:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:31:22,295 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:28.874906+00:00 (in 6.580029 seconds) chunkcleanupworker stdout | 2023-08-08 06:31:22,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:32 UTC)" (scheduled at 2023-08-08 06:31:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:31:22,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:31:22,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 22, 295362), True, datetime.datetime(2023, 8, 8, 6, 31, 22, 295362), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:31:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:31:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:31:22,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:31:24,622 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} builder stdout | 2023-08-08 06:31:25,344 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:31:25,353 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:31:25,353 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:31:25,357 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:31:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:35.371682+00:00 (in 9.999421 seconds) notificationworker stdout | 2023-08-08 06:31:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:35 UTC)" (scheduled at 2023-08-08 06:31:25.371682+00:00) notificationworker stdout | 2023-08-08 06:31:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:31:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 25, 372761), True, datetime.datetime(2023, 8, 8, 6, 31, 25, 372761), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:31:25,379 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:31:25,379 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:25,379 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:35 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:31:25,416 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:31:25,419 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:31:25,422 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} exportactionlogsworker stdout | 2023-08-08 06:31:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:31:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:47.398551+00:00 (in 21.883708 seconds) exportactionlogsworker stdout | 2023-08-08 06:31:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:25 UTC)" (scheduled at 2023-08-08 06:31:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:31:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:31:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:31:26,499 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:31:26,889 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:31:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:31:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:49.687363+00:00 (in 22.796108 seconds) securityworker stdout | 2023-08-08 06:31:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:56 UTC)" (scheduled at 2023-08-08 06:31:26.890741+00:00) securityworker stdout | 2023-08-08 06:31:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:31:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:31:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:31:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:31:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:31:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:31:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state repositorygcworker stdout | 2023-08-08 06:31:26,894 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} securityworker stdout | 2023-08-08 06:31:26,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:31:26,898 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:31:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:31:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 26, 26, 898148), 1, 2]) securityworker stdout | 2023-08-08 06:31:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:26,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 26, 26, 898148), 1, 2]) securityworker stdout | 2023-08-08 06:31:26,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:26,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:26,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:31:26,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:31:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:31:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:31:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:31:26,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:31:26,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:31:26,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:31:56 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:31:27,811 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:31:27,964 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:31:28,047 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:31:28,155 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:31:28,160 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:31:28,164 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:31:28,485 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:31:28,704 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:31:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:31:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:32.293757+00:00 (in 3.418369 seconds) chunkcleanupworker stdout | 2023-08-08 06:31:28,876 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:28 UTC)" (scheduled at 2023-08-08 06:31:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:31:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:31:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:31:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:31:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:39.417660+00:00 (in 9.999502 seconds) gcworker stdout | 2023-08-08 06:31:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:39 UTC)" (scheduled at 2023-08-08 06:31:29.417660+00:00) gcworker stdout | 2023-08-08 06:31:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266689418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:31:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:31:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:31:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:31:30,027 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:31:30,149 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:31:31,185 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} securityscanningnotificationworker stdout | 2023-08-08 06:31:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:31:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:54.284093+00:00 (in 22.999730 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:31:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:36:31 UTC)" (scheduled at 2023-08-08 06:31:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:31:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 31, 31, 284810), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:31:31,292 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 31, 31, 284810), True, datetime.datetime(2023, 8, 8, 6, 31, 31, 284810), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:31:31,294 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 31, 31, 284810), True, datetime.datetime(2023, 8, 8, 6, 31, 31, 284810), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 31, 31, 284810), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:31:31,295 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:31:31,296 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:36:31 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:31:32,108 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:31:32,250 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:31:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:31:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:42.293757+00:00 (in 9.999522 seconds) chunkcleanupworker stdout | 2023-08-08 06:31:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:42 UTC)" (scheduled at 2023-08-08 06:31:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:31:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:31:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 32, 294648), True, datetime.datetime(2023, 8, 8, 6, 31, 32, 294648), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:31:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:31:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:31:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:42 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:31:32,629 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:31:33,114 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:31:33,401 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:31:33,747 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:31:33,754 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:31:33,756 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:31:33,759 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:31:33,764 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:31:33,774 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:31:33,779 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:31:33,786 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:31:33,789 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:31:33,866 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} builder stdout | 2023-08-08 06:31:35,357 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:31:35,367 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:31:35,367 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:31:35,371 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:31:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:45.371682+00:00 (in 9.999468 seconds) notificationworker stdout | 2023-08-08 06:31:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:45 UTC)" (scheduled at 2023-08-08 06:31:35.371682+00:00) notificationworker stdout | 2023-08-08 06:31:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:31:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 35, 372736), True, datetime.datetime(2023, 8, 8, 6, 31, 35, 372736), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:31:35,379 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:31:35,379 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:35,379 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:31:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:31:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:54.393627+00:00 (in 17.813252 seconds) namespacegcworker stdout | 2023-08-08 06:31:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:36 UTC)" (scheduled at 2023-08-08 06:31:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:31:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:31:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:31:37,179 [197] [DEBUG] [app] Starting request: urn:request:75567a30-c58b-42b4-86e6-bf4ee0bcc630 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:31:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:37,180 [200] [DEBUG] [app] Starting request: urn:request:bf0af96b-955f-4589-8c10-544f9ead7044 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:31:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:31:37,185 [209] [DEBUG] [app] Starting request: urn:request:a79e1785-af66-4b79-be0e-be8d7273645d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:31:37,185 [204] [DEBUG] [app] Starting request: urn:request:303ca63b-d0b6-4ef0-9d4f-ced38be66f15 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:31:37,186 [209] [DEBUG] [app] Ending request: urn:request:a79e1785-af66-4b79-be0e-be8d7273645d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a79e1785-af66-4b79-be0e-be8d7273645d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:31:37,186 [204] [DEBUG] [app] Ending request: urn:request:303ca63b-d0b6-4ef0-9d4f-ced38be66f15 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:303ca63b-d0b6-4ef0-9d4f-ced38be66f15', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:31:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:31:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:31:37,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:31:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:37,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:37,191 [197] [DEBUG] [app] Starting request: urn:request:79f6ecab-3179-42ef-bc9f-8e13b903bafb (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:37,192 [197] [DEBUG] [app] Ending request: urn:request:79f6ecab-3179-42ef-bc9f-8e13b903bafb (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:79f6ecab-3179-42ef-bc9f-8e13b903bafb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:31:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:31:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:37,192 [199] [DEBUG] [app] Starting request: urn:request:555c333c-8382-463d-93d8-1e27fca8d07d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:31:37,193 [199] [DEBUG] [app] Ending request: urn:request:555c333c-8382-463d-93d8-1e27fca8d07d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:555c333c-8382-463d-93d8-1e27fca8d07d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:31:37,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:31:37,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:37,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:31:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:31:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:07.197412+00:00 (in 29.999487 seconds) gunicorn-web stdout | 2023-08-08 06:31:37,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:31:37,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:31:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:07 UTC)" (scheduled at 2023-08-08 06:31:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:31:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 31, 37, 198768), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:31:37,204 [197] [DEBUG] [app] Ending request: urn:request:75567a30-c58b-42b4-86e6-bf4ee0bcc630 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:75567a30-c58b-42b4-86e6-bf4ee0bcc630', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:31:37,204 [200] [DEBUG] [app] Ending request: urn:request:bf0af96b-955f-4589-8c10-544f9ead7044 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bf0af96b-955f-4589-8c10-544f9ead7044', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:31:37,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:31:37,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:31:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:31:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:31:37,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:31:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:31:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:31:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:31:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:31:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:31:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:31:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:31:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:31:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:31:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:49.417660+00:00 (in 9.999447 seconds) gcworker stdout | 2023-08-08 06:31:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:49 UTC)" (scheduled at 2023-08-08 06:31:39.417660+00:00) gcworker stdout | 2023-08-08 06:31:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266699418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:31:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:31:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:31:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:31:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:31:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:52.293757+00:00 (in 9.999440 seconds) chunkcleanupworker stdout | 2023-08-08 06:31:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:52 UTC)" (scheduled at 2023-08-08 06:31:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:31:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:31:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 42, 294764), True, datetime.datetime(2023, 8, 8, 6, 31, 42, 294764), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:31:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:31:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:31:42,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:31:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:31:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:55.371682+00:00 (in 9.998988 seconds) notificationworker stdout | 2023-08-08 06:31:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:55 UTC)" (scheduled at 2023-08-08 06:31:45.371682+00:00) notificationworker stdout | 2023-08-08 06:31:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:31:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 45, 373182), True, datetime.datetime(2023, 8, 8, 6, 31, 45, 373182), 0, 'notification/%', 50, 1, 0]) builder stdout | 2023-08-08 06:31:45,377 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue notificationworker stdout | 2023-08-08 06:31:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:31:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:55 UTC)" executed successfully builder stdout | 2023-08-08 06:31:45,385 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:31:45,386 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:31:45,390 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:31:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:31:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:25.514341+00:00 (in 38.115318 seconds) exportactionlogsworker stdout | 2023-08-08 06:31:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:47 UTC)" (scheduled at 2023-08-08 06:31:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:31:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:31:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 47, 399501), True, datetime.datetime(2023, 8, 8, 6, 31, 47, 399501), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:31:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:31:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:31:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:31:49,329 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:31:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:31:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:59.417660+00:00 (in 9.999466 seconds) gcworker stdout | 2023-08-08 06:31:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:59 UTC)" (scheduled at 2023-08-08 06:31:49.417660+00:00) gcworker stdout | 2023-08-08 06:31:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266709418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:31:49,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:31:49,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:31:49,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:31:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:31:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:31:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:31:56.890741+00:00 (in 7.202788 seconds) securityworker stdout | 2023-08-08 06:31:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:19 UTC)" (scheduled at 2023-08-08 06:31:49.687363+00:00) securityworker stdout | 2023-08-08 06:31:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:31:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:31:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:31:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:31:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:31:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:31:50,439 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:31:51,578 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:31:51,978 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:31:52,179 [200] [DEBUG] [app] Starting request: urn:request:77fad3d1-d96f-4182-93a4-73e0e7b0876b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:31:52,179 [197] [DEBUG] [app] Starting request: urn:request:41e8bc27-1f2a-4111-9d00-2e93643730c2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:31:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:31:52,185 [203] [DEBUG] [app] Starting request: urn:request:c01016f3-9554-4782-bb35-b4b47e97dc17 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:31:52,186 [209] [DEBUG] [app] Starting request: urn:request:ce41712f-ea77-4aa1-bc18-7a8cf5a5d5f6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:31:52,186 [203] [DEBUG] [app] Ending request: urn:request:c01016f3-9554-4782-bb35-b4b47e97dc17 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c01016f3-9554-4782-bb35-b4b47e97dc17', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:31:52,186 [209] [DEBUG] [app] Ending request: urn:request:ce41712f-ea77-4aa1-bc18-7a8cf5a5d5f6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ce41712f-ea77-4aa1-bc18-7a8cf5a5d5f6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:31:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:31:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 06:31:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:52,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:31:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:52,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:31:52,193 [198] [DEBUG] [app] Starting request: urn:request:a49c0322-52db-466e-b5f9-516c335df172 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:52,193 [198] [DEBUG] [app] Ending request: urn:request:a49c0322-52db-466e-b5f9-516c335df172 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a49c0322-52db-466e-b5f9-516c335df172', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:31:52,193 [199] [DEBUG] [app] Starting request: urn:request:fd775c02-323c-4810-9da7-833eb4128687 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:31:52,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:31:52,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:52,194 [199] [DEBUG] [app] Ending request: urn:request:fd775c02-323c-4810-9da7-833eb4128687 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fd775c02-323c-4810-9da7-833eb4128687', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:31:52,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:31:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:31:52,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:31:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.003) gunicorn-web stdout | 2023-08-08 06:31:52,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:31:52,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:31:52,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:31:52,200 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:31:52,205 [197] [DEBUG] [app] Ending request: urn:request:41e8bc27-1f2a-4111-9d00-2e93643730c2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:41e8bc27-1f2a-4111-9d00-2e93643730c2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:31:52,205 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:31:52,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:31:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:31:52,205 [200] [DEBUG] [app] Ending request: urn:request:77fad3d1-d96f-4182-93a4-73e0e7b0876b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:77fad3d1-d96f-4182-93a4-73e0e7b0876b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:31:52,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:31:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) gunicorn-web stdout | 2023-08-08 06:31:52,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:31:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:31:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) chunkcleanupworker stdout | 2023-08-08 06:31:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:31:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:02.293757+00:00 (in 9.999454 seconds) chunkcleanupworker stdout | 2023-08-08 06:31:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:02 UTC)" (scheduled at 2023-08-08 06:31:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:31:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:31:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 52, 294663), True, datetime.datetime(2023, 8, 8, 6, 31, 52, 294663), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:31:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:31:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:31:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:31:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:31:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:13.097375+00:00 (in 18.812648 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:31:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:54 UTC)" (scheduled at 2023-08-08 06:31:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:31:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:31:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:31:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:31:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:36.579841+00:00 (in 42.185727 seconds) namespacegcworker stdout | 2023-08-08 06:31:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:54 UTC)" (scheduled at 2023-08-08 06:31:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:31:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:31:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 54, 394534), True, datetime.datetime(2023, 8, 8, 6, 31, 54, 394534), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:31:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:31:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:31:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:32:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:31:54,656 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:31:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:31:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:02.178530+00:00 (in 6.806264 seconds) notificationworker stdout | 2023-08-08 06:31:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:05 UTC)" (scheduled at 2023-08-08 06:31:55.371682+00:00) notificationworker stdout | 2023-08-08 06:31:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:31:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 31, 55, 372908), True, datetime.datetime(2023, 8, 8, 6, 31, 55, 372908), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:31:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:31:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:31:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:05 UTC)" executed successfully builder stdout | 2023-08-08 06:31:55,395 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:31:55,403 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:31:55,403 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:31:55,408 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-secscan stdout | 2023-08-08 06:31:55,425 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:31:55,428 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:31:55,433 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} blobuploadcleanupworker stdout | 2023-08-08 06:31:56,512 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:31:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:31:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:19.687363+00:00 (in 22.795659 seconds) securityworker stdout | 2023-08-08 06:31:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:26 UTC)" (scheduled at 2023-08-08 06:31:56.890741+00:00) securityworker stdout | 2023-08-08 06:31:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:31:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:31:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:31:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:31:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:31:56,894 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:31:56,894 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:31:56,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:31:56,898 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:31:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) chunkcleanupworker stdout | 2023-08-08 06:31:56,906 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} securityworker stdout | 2023-08-08 06:31:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 26, 56, 897812), 1, 2]) securityworker stdout | 2023-08-08 06:31:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:56,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 26, 56, 897812), 1, 2]) securityworker stdout | 2023-08-08 06:31:56,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:31:56,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:31:56,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:31:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:31:56,911 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:31:56,911 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:31:56,911 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:31:56,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:31:56,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:31:56,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:26 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:31:56,915 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:31:57,820 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:31:58,001 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:31:58,060 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:31:58,169 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:31:58,173 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:31:58,176 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:31:58,505 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:31:58,714 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:31:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:31:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:09.417660+00:00 (in 9.999389 seconds) gcworker stdout | 2023-08-08 06:31:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:09 UTC)" (scheduled at 2023-08-08 06:31:59.417660+00:00) gcworker stdout | 2023-08-08 06:31:59,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266719419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:31:59,430 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:31:59,430 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:31:59,430 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:32:00,047 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:32:00,187 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:32:01,193 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:32:02,146 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} notificationworker stdout | 2023-08-08 06:32:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:32:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:05.371682+00:00 (in 3.192606 seconds) notificationworker stdout | 2023-08-08 06:32:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:02 UTC)" (scheduled at 2023-08-08 06:32:02.178530+00:00) notificationworker stdout | 2023-08-08 06:32:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:32:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:02 UTC)" executed successfully servicekey stdout | 2023-08-08 06:32:02,282 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:32:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:32:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:12.293757+00:00 (in 9.999504 seconds) chunkcleanupworker stdout | 2023-08-08 06:32:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:12 UTC)" (scheduled at 2023-08-08 06:32:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:32:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:32:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 2, 294806), True, datetime.datetime(2023, 8, 8, 6, 32, 2, 294806), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:32:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:32:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:32:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:12 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:32:02,641 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:32:03,152 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:32:03,435 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:32:03,760 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:32:03,763 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:32:03,768 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:32:03,772 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:32:03,776 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:32:03,785 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:32:03,789 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:32:03,797 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:32:03,800 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:32:03,904 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} repositorygcworker stdout | 2023-08-08 06:32:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:32:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:09.793658+00:00 (in 5.605644 seconds) repositorygcworker stdout | 2023-08-08 06:32:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:04 UTC)" (scheduled at 2023-08-08 06:32:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:32:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:32:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:32:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:32:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:15.371682+00:00 (in 9.999421 seconds) notificationworker stdout | 2023-08-08 06:32:05,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:15 UTC)" (scheduled at 2023-08-08 06:32:05.371682+00:00) notificationworker stdout | 2023-08-08 06:32:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:32:05,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 5, 373552), True, datetime.datetime(2023, 8, 8, 6, 32, 5, 373552), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:32:05,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:32:05,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:32:05,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:15 UTC)" executed successfully builder stdout | 2023-08-08 06:32:05,418 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:32:05,427 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:32:05,427 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:32:05,432 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:32:07,179 [199] [DEBUG] [app] Starting request: urn:request:3850aebd-2d6c-454b-b81b-60dd79a9e73f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:32:07,180 [200] [DEBUG] [app] Starting request: urn:request:0051ca18-a8a0-48fd-9ad7-7abe86aaa698 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:32:07,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:07,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:07,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:07,186 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:32:07,186 [208] [DEBUG] [app] Starting request: urn:request:9f08a0f9-4316-40ff-a574-fb4b1cf630ac (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:32:07,187 [206] [DEBUG] [app] Starting request: urn:request:e6589dd1-d98e-4199-8528-f2fa28226209 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:32:07,187 [208] [DEBUG] [app] Ending request: urn:request:9f08a0f9-4316-40ff-a574-fb4b1cf630ac (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9f08a0f9-4316-40ff-a574-fb4b1cf630ac', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:32:07,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:32:07,187 [206] [DEBUG] [app] Ending request: urn:request:e6589dd1-d98e-4199-8528-f2fa28226209 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e6589dd1-d98e-4199-8528-f2fa28226209', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 06:32:07,188 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:32:07,188 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:32:07,188 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:07,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:07,190 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:07,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:07,192 [200] [DEBUG] [app] Starting request: urn:request:81685dd7-869c-4e65-bcaf-551fe3088a62 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:32:07,193 [200] [DEBUG] [app] Ending request: urn:request:81685dd7-869c-4e65-bcaf-551fe3088a62 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:81685dd7-869c-4e65-bcaf-551fe3088a62', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:32:07,193 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:32:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:07,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:32:07,194 [198] [DEBUG] [app] Starting request: urn:request:92eb2e49-2d80-4ddd-b8bd-9695c2e1abb4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:32:07,194 [198] [DEBUG] [app] Ending request: urn:request:92eb2e49-2d80-4ddd-b8bd-9695c2e1abb4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:92eb2e49-2d80-4ddd-b8bd-9695c2e1abb4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:32:07,195 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:32:07,195 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:07,196 [199] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:32:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:32:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:37.197412+00:00 (in 29.999539 seconds) buildlogsarchiver stdout | 2023-08-08 06:32:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:37 UTC)" (scheduled at 2023-08-08 06:32:07.197412+00:00) gunicorn-web stdout | 2023-08-08 06:32:07,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:32:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 32, 7, 198338), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:07,201 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:32:07,204 [200] [DEBUG] [app] Ending request: urn:request:0051ca18-a8a0-48fd-9ad7-7abe86aaa698 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0051ca18-a8a0-48fd-9ad7-7abe86aaa698', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:32:07,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:32:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:32:07,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:32:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:32:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:32:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 gunicorn-web stdout | 2023-08-08 06:32:07,207 [199] [DEBUG] [app] Ending request: urn:request:3850aebd-2d6c-454b-b81b-60dd79a9e73f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3850aebd-2d6c-454b-b81b-60dd79a9e73f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:32:07,207 [199] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:32:07,207 [56] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:32:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.029) gunicorn-web stdout | 2023-08-08 06:32:07,207 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:32:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:32:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:32:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:32:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:32:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:19.417660+00:00 (in 9.999167 seconds) gcworker stdout | 2023-08-08 06:32:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:19 UTC)" (scheduled at 2023-08-08 06:32:09.417660+00:00) gcworker stdout | 2023-08-08 06:32:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266729418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:32:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:32:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:32:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:19 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:32:09,640 [197] [DEBUG] [app] Starting request: urn:request:ee2b881d-2b88-4240-b4b7-a818e94260da (/api/v1/repository/user1_build_test/github/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:32:09,640 [200] [DEBUG] [app] Starting request: urn:request:69c444cc-a744-4f56-8eb4-5a18b12a2bcd (/api/v1/repository/user1_build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:32:09,641 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,641 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,641 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,641 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,641 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:09,641 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:09,648 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:32:09,648 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:32:09,648 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,648 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:32:09,649 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,649 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:09,649 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:09,649 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:09,649 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:32:09,649 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:32:09,649 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,649 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:32:09,649 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,650 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:09,650 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:09,650 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:09,651 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:32:09,652 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:32:09,658 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:32:09,658 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:32:09,658 [200] [DEBUG] [endpoints.api.repository] Get repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:32:09,659 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:09,659 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:09,661 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [4, 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:09,662 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:09,662 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [4, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:32:09,663 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,664 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:09,664 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [4, None, 1691476329663, False, 101, 0]) gunicorn-web stdout | 2023-08-08 06:32:09,665 [200] [DEBUG] [app] Ending request: urn:request:69c444cc-a744-4f56-8eb4-5a18b12a2bcd (/api/v1/repository/user1_build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:69c444cc-a744-4f56-8eb4-5a18b12a2bcd', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false', 'path': '/api/v1/repository/user1_build_test/github', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:32:09,665 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:32:09,666 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:32:09 +0000] "GET /api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false HTTP/1.0" 200 3632 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:32:09 +0000] "GET /api/v1/repository/user1_build_test/github?includeStats=true&includeTags=false HTTP/2.0" 200 3632 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 92 0.027) gunicorn-web stdout | 2023-08-08 06:32:09,666 [197] [DEBUG] [app] Ending request: urn:request:ee2b881d-2b88-4240-b4b7-a818e94260da (/api/v1/repository/user1_build_test/github/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:ee2b881d-2b88-4240-b4b7-a818e94260da', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/user1_build_test/github/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:32:09,667 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:32:09 +0000] "GET /api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 887 0.029) gunicorn-web stdout | 2023-08-08 06:32:09,668 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:32:09 +0000] "GET /api/v1/repository/user1_build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" repositorygcworker stdout | 2023-08-08 06:32:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:32:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:04.187429+00:00 (in 54.393090 seconds) repositorygcworker stdout | 2023-08-08 06:32:09,795 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:09 UTC)" (scheduled at 2023-08-08 06:32:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:32:09,795 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:32:09,796 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 9, 795427), True, datetime.datetime(2023, 8, 8, 6, 32, 9, 795427), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:32:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:32:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:32:09,804 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:09 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:32:10,536 [200] [DEBUG] [app] Starting request: urn:request:a6bff418-8bb4-4de1-89c6-b6b0541f70f4 (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:32:10,536 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:10,536 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:10,537 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:10,544 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:32:10,545 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:32:10,545 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:10,545 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:32:10,545 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:10,545 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:10,545 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:10,545 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:10,547 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:32:10,553 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:32:10,555 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:10,558 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:32:10,562 [200] [DEBUG] [app] Ending request: urn:request:a6bff418-8bb4-4de1-89c6-b6b0541f70f4 (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:a6bff418-8bb4-4de1-89c6-b6b0541f70f4', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=3', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:32:10,563 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:32:10 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 419 0.029) gunicorn-web stdout | 2023-08-08 06:32:10,564 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:32:10 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:32:12,157 [200] [DEBUG] [app] Starting request: urn:request:1df8ce75-4b23-4d09-a1a6-01842f97bb31 (/api/v1/repository/user1_build_test/github/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:32:12,157 [197] [DEBUG] [app] Starting request: urn:request:184dfe82-4deb-4a55-89fc-113e355a7413 (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:32:12,157 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:12,157 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:12,157 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:12,157 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:12,158 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:12,158 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:12,165 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:32:12,165 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:32:12,165 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:12,166 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:32:12,166 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:12,166 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:32:12,166 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:12,166 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:12,166 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:12,166 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:32:12,166 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:12,166 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:32:12,166 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:12,166 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:12,166 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:12,166 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:12,168 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:32:12,169 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:32:12,175 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:32:12,175 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:32:12,176 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:12,176 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:12,179 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['user1_build_test', 'github']) gunicorn-web stdout | 2023-08-08 06:32:12,180 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 10]) gunicorn-web stdout | 2023-08-08 06:32:12,181 [200] [DEBUG] [app] Ending request: urn:request:1df8ce75-4b23-4d09-a1a6-01842f97bb31 (/api/v1/repository/user1_build_test/github/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:1df8ce75-4b23-4d09-a1a6-01842f97bb31', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/', 'path': '/api/v1/repository/user1_build_test/github/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:32:12,182 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:32:12,182 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:32:12 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/ HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:32:12 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/ HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 510 0.025) gunicorn-web stdout | 2023-08-08 06:32:12,183 [197] [DEBUG] [app] Ending request: urn:request:184dfe82-4deb-4a55-89fc-113e355a7413 (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:184dfe82-4deb-4a55-89fc-113e355a7413', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=10', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:32:12,184 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:32:12 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 78 0.028) gunicorn-web stdout | 2023-08-08 06:32:12,185 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:32:12 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" chunkcleanupworker stdout | 2023-08-08 06:32:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:32:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:22.293757+00:00 (in 9.999457 seconds) chunkcleanupworker stdout | 2023-08-08 06:32:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:22 UTC)" (scheduled at 2023-08-08 06:32:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:32:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:32:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 12, 294783), True, datetime.datetime(2023, 8, 8, 6, 32, 12, 294783), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:32:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:32:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:32:12,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:32:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:32:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:54.284093+00:00 (in 41.186201 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:32:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:13 UTC)" (scheduled at 2023-08-08 06:32:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:32:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:32:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 13, 98363), True, datetime.datetime(2023, 8, 8, 6, 32, 13, 98363), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:32:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:32:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:32:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:32:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:32:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:25.371682+00:00 (in 9.999429 seconds) notificationworker stdout | 2023-08-08 06:32:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:25 UTC)" (scheduled at 2023-08-08 06:32:15.371682+00:00) notificationworker stdout | 2023-08-08 06:32:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:32:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 15, 372835), True, datetime.datetime(2023, 8, 8, 6, 32, 15, 372835), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:32:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:32:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:32:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:25 UTC)" executed successfully builder stdout | 2023-08-08 06:32:15,443 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:32:15,452 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:32:15,452 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:32:15,457 [55] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:32:19,365 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:32:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:32:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:29.417660+00:00 (in 9.999427 seconds) gcworker stdout | 2023-08-08 06:32:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:29 UTC)" (scheduled at 2023-08-08 06:32:19.417660+00:00) gcworker stdout | 2023-08-08 06:32:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266739418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:32:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:32:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:32:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:32:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:32:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:26.890741+00:00 (in 7.202884 seconds) securityworker stdout | 2023-08-08 06:32:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:49 UTC)" (scheduled at 2023-08-08 06:32:19.687363+00:00) securityworker stdout | 2023-08-08 06:32:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:32:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:32:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:32:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:32:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:32:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:32:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 27, 19, 691809), 1, 2]) securityworker stdout | 2023-08-08 06:32:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 27, 19, 691809), 1, 2]) securityworker stdout | 2023-08-08 06:32:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:32:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:32:20,449 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:32:21,594 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:32:22,017 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:32:22,180 [200] [DEBUG] [app] Starting request: urn:request:581f6a83-4b08-416d-a5d2-00458457ad2c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:32:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:22,181 [199] [DEBUG] [app] Starting request: urn:request:d4e12dc7-9b01-4834-8c8c-0828db51b8a3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:32:22,183 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:22,186 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:32:22,186 [203] [DEBUG] [app] Starting request: urn:request:e3d65b19-374c-46f5-8908-ca53107fc174 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:32:22,187 [203] [DEBUG] [app] Ending request: urn:request:e3d65b19-374c-46f5-8908-ca53107fc174 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e3d65b19-374c-46f5-8908-ca53107fc174', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:32:22,187 [209] [DEBUG] [app] Starting request: urn:request:9fb320c8-3d1b-4e06-9525-4f89a734e746 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:32:22,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:32:22,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:32:22,187 [209] [DEBUG] [app] Ending request: urn:request:9fb320c8-3d1b-4e06-9525-4f89a734e746 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9fb320c8-3d1b-4e06-9525-4f89a734e746', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:32:22,188 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:32:22,188 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:22,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:22,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:22,192 [199] [DEBUG] [app] Starting request: urn:request:38c798cc-161f-4496-a147-a0fbca9495ff (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:32:22,193 [199] [DEBUG] [app] Ending request: urn:request:38c798cc-161f-4496-a147-a0fbca9495ff (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:38c798cc-161f-4496-a147-a0fbca9495ff', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:32:22,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:32:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:22,193 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:32:22,194 [199] [DEBUG] [app] Starting request: urn:request:2d9acbe1-4269-414a-afc4-2abb88e2cdce (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:32:22,195 [199] [DEBUG] [app] Ending request: urn:request:2d9acbe1-4269-414a-afc4-2abb88e2cdce (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2d9acbe1-4269-414a-afc4-2abb88e2cdce', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:32:22,195 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:32:22,195 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:22,196 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:32:22,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:32:22,200 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:32:22,205 [200] [DEBUG] [app] Ending request: urn:request:581f6a83-4b08-416d-a5d2-00458457ad2c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:581f6a83-4b08-416d-a5d2-00458457ad2c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:32:22,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:32:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:32:22,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:32:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:32:22,206 [199] [DEBUG] [app] Ending request: urn:request:d4e12dc7-9b01-4834-8c8c-0828db51b8a3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d4e12dc7-9b01-4834-8c8c-0828db51b8a3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:32:22,206 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:32:22,206 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:32:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:32:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:32:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:32:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:28.874906+00:00 (in 6.580592 seconds) chunkcleanupworker stdout | 2023-08-08 06:32:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:32 UTC)" (scheduled at 2023-08-08 06:32:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:32:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:32:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 22, 294887), True, datetime.datetime(2023, 8, 8, 6, 32, 22, 294887), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:32:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:32:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:32:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:32:24,692 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:32:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:32:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:35.371682+00:00 (in 9.999437 seconds) notificationworker stdout | 2023-08-08 06:32:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:35 UTC)" (scheduled at 2023-08-08 06:32:25.371682+00:00) notificationworker stdout | 2023-08-08 06:32:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:32:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 25, 372777), True, datetime.datetime(2023, 8, 8, 6, 32, 25, 372777), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:32:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:32:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:32:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:35 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:32:25,433 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:32:25,437 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:32:25,440 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:32:25,460 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:32:25,469 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:32:25,469 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:32:25,474 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:32:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:32:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:47.398551+00:00 (in 21.883569 seconds) exportactionlogsworker stdout | 2023-08-08 06:32:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:25 UTC)" (scheduled at 2023-08-08 06:32:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:32:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:32:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:25 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:32:26,543 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:32:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:32:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:49.687363+00:00 (in 22.796095 seconds) securityworker stdout | 2023-08-08 06:32:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:56 UTC)" (scheduled at 2023-08-08 06:32:26.890741+00:00) securityworker stdout | 2023-08-08 06:32:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:32:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:32:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:32:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:32:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:32:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:32:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:32:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:32:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:32:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:32:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 27, 26, 897000), 1, 2]) securityworker stdout | 2023-08-08 06:32:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 27, 26, 897000), 1, 2]) securityworker stdout | 2023-08-08 06:32:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:32:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:32:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:32:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:32:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:32:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:32:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:32:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:32:56 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:32:26,932 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:32:26,953 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:32:27,830 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:32:28,039 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:32:28,073 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:32:28,183 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:32:28,186 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:32:28,189 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:32:28,538 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:32:28,745 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:32:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:32:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:32.293757+00:00 (in 3.418260 seconds) chunkcleanupworker stdout | 2023-08-08 06:32:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:28 UTC)" (scheduled at 2023-08-08 06:32:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:32:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:32:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:32:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:32:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:39.417660+00:00 (in 9.999462 seconds) gcworker stdout | 2023-08-08 06:32:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:39 UTC)" (scheduled at 2023-08-08 06:32:29.417660+00:00) gcworker stdout | 2023-08-08 06:32:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266749418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:32:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:32:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:32:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:32:30,072 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:32:30,225 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:32:31,223 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:32:32,178 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} chunkcleanupworker stdout | 2023-08-08 06:32:32,295 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:32:32,295 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:42.293757+00:00 (in 9.998534 seconds) chunkcleanupworker stdout | 2023-08-08 06:32:32,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:42 UTC)" (scheduled at 2023-08-08 06:32:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:32:32,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:32:32,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 32, 295847), True, datetime.datetime(2023, 8, 8, 6, 32, 32, 295847), 0, 'chunk_cleanup/%', 50, 1, 0]) servicekey stdout | 2023-08-08 06:32:32,298 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} chunkcleanupworker stdout | 2023-08-08 06:32:32,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:32:32,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:32:32,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:42 UTC)" executed successfully globalpromstats stdout | 2023-08-08 06:32:32,675 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:32:33,191 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:32:33,473 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:32:33,774 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:32:33,777 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:32:33,780 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:32:33,784 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:32:33,788 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:32:33,796 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:32:33,799 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:32:33,808 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:32:33,813 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:32:33,936 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:32:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:32:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:45.371682+00:00 (in 9.999392 seconds) notificationworker stdout | 2023-08-08 06:32:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:45 UTC)" (scheduled at 2023-08-08 06:32:35.371682+00:00) notificationworker stdout | 2023-08-08 06:32:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:32:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 35, 372938), True, datetime.datetime(2023, 8, 8, 6, 32, 35, 372938), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:32:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:32:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:32:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:45 UTC)" executed successfully builder stdout | 2023-08-08 06:32:35,484 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:32:35,495 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:32:35,495 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:32:35,499 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:32:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:32:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:54.393627+00:00 (in 17.813176 seconds) namespacegcworker stdout | 2023-08-08 06:32:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:36 UTC)" (scheduled at 2023-08-08 06:32:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:32:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:32:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:32:37,179 [199] [DEBUG] [app] Starting request: urn:request:1fcdca6f-79a1-4441-87d3-152259b10b6b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:32:37,181 [198] [DEBUG] [app] Starting request: urn:request:0ff57295-4a28-4cdf-be29-876ee26f4bb6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:32:37,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:37,182 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:37,185 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:37,186 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:32:37,186 [203] [DEBUG] [app] Starting request: urn:request:8c6b62c4-3002-4aa7-994c-7b7bb4918c3a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:32:37,187 [209] [DEBUG] [app] Starting request: urn:request:3e4b854a-ebb8-4e47-b088-42755941724f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:32:37,187 [203] [DEBUG] [app] Ending request: urn:request:8c6b62c4-3002-4aa7-994c-7b7bb4918c3a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8c6b62c4-3002-4aa7-994c-7b7bb4918c3a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:32:37,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:32:37,188 [209] [DEBUG] [app] Ending request: urn:request:3e4b854a-ebb8-4e47-b088-42755941724f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3e4b854a-ebb8-4e47-b088-42755941724f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:32:37,188 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:32:37,188 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:32:37,188 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:37,189 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:37,190 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:37,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:37,193 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:37,193 [197] [DEBUG] [app] Starting request: urn:request:027817da-e570-4e82-bc99-7c4a0fb01d63 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:32:37,194 [197] [DEBUG] [app] Ending request: urn:request:027817da-e570-4e82-bc99-7c4a0fb01d63 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:027817da-e570-4e82-bc99-7c4a0fb01d63', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:32:37,194 [199] [DEBUG] [app] Starting request: urn:request:74c10fd4-4653-47c4-9118-bae23bf93460 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:32:37,194 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:32:37,195 [199] [DEBUG] [app] Ending request: urn:request:74c10fd4-4653-47c4-9118-bae23bf93460 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:74c10fd4-4653-47c4-9118-bae23bf93460', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:32:37,195 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:32:37,195 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:37,195 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:37,196 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:32:37,196 [199] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:32:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:32:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:07.197412+00:00 (in 29.999521 seconds) buildlogsarchiver stdout | 2023-08-08 06:32:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:07 UTC)" (scheduled at 2023-08-08 06:32:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:32:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 32, 37, 198319), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:37,201 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:32:37,201 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:32:37,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:32:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 gunicorn-web stdout | 2023-08-08 06:32:37,207 [198] [DEBUG] [app] Ending request: urn:request:0ff57295-4a28-4cdf-be29-876ee26f4bb6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0ff57295-4a28-4cdf-be29-876ee26f4bb6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:32:37,207 [199] [DEBUG] [app] Ending request: urn:request:1fcdca6f-79a1-4441-87d3-152259b10b6b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1fcdca6f-79a1-4441-87d3-152259b10b6b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:32:37,207 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:32:37,207 [199] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:32:37,207 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:32:37,207 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:32:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:32:37,207 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:32:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:32:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.030 47 0.030) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:32:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.028) buildlogsarchiver stdout | 2023-08-08 06:32:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:32:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:32:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:32:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:49.417660+00:00 (in 9.999443 seconds) gcworker stdout | 2023-08-08 06:32:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:49 UTC)" (scheduled at 2023-08-08 06:32:39.417660+00:00) gcworker stdout | 2023-08-08 06:32:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266759418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:32:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:32:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:32:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:32:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:32:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:52.293757+00:00 (in 9.999503 seconds) chunkcleanupworker stdout | 2023-08-08 06:32:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:52 UTC)" (scheduled at 2023-08-08 06:32:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:32:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:32:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 42, 294640), True, datetime.datetime(2023, 8, 8, 6, 32, 42, 294640), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:32:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:32:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:32:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:32:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:32:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:55.371682+00:00 (in 9.999494 seconds) notificationworker stdout | 2023-08-08 06:32:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:55 UTC)" (scheduled at 2023-08-08 06:32:45.371682+00:00) notificationworker stdout | 2023-08-08 06:32:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:32:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 45, 372556), True, datetime.datetime(2023, 8, 8, 6, 32, 45, 372556), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:32:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:32:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:32:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:55 UTC)" executed successfully builder stdout | 2023-08-08 06:32:45,505 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:32:45,513 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:32:45,513 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:32:45,518 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:32:47,399 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:32:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:25.514341+00:00 (in 38.115196 seconds) exportactionlogsworker stdout | 2023-08-08 06:32:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:47 UTC)" (scheduled at 2023-08-08 06:32:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:32:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:32:47,401 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 47, 399603), True, datetime.datetime(2023, 8, 8, 6, 32, 47, 399603), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:32:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:32:47,409 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:32:47,409 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:32:49,386 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:32:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:32:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:59.417660+00:00 (in 9.999445 seconds) gcworker stdout | 2023-08-08 06:32:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:59 UTC)" (scheduled at 2023-08-08 06:32:49.417660+00:00) gcworker stdout | 2023-08-08 06:32:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266769418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:32:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:32:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:32:49,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:32:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:32:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:32:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:32:56.890741+00:00 (in 7.202645 seconds) securityworker stdout | 2023-08-08 06:32:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:19 UTC)" (scheduled at 2023-08-08 06:32:49.687363+00:00) securityworker stdout | 2023-08-08 06:32:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:32:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:32:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:32:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:32:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:32:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:32:50,477 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:32:51,628 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:32:52,054 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:32:52,178 [200] [DEBUG] [app] Starting request: urn:request:64ee8489-cbff-4138-aa70-1dcd1c068437 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:32:52,179 [198] [DEBUG] [app] Starting request: urn:request:7391d500-61c0-4dbc-9fff-a8795a13e0d9 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:32:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:52,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:32:52,184 [209] [DEBUG] [app] Starting request: urn:request:dc85e504-c92b-45e7-93db-36b12d10b82e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:32:52,184 [210] [DEBUG] [app] Starting request: urn:request:242dc99c-78b6-4241-84ae-ea223803e6b7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:32:52,184 [209] [DEBUG] [app] Ending request: urn:request:dc85e504-c92b-45e7-93db-36b12d10b82e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:dc85e504-c92b-45e7-93db-36b12d10b82e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:32:52,185 [210] [DEBUG] [app] Ending request: urn:request:242dc99c-78b6-4241-84ae-ea223803e6b7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:242dc99c-78b6-4241-84ae-ea223803e6b7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:32:52,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:32:52,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:32:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:32:52,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:52,186 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:32:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:52,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:32:52,190 [197] [DEBUG] [app] Starting request: urn:request:083a7742-065f-48ca-9c73-cb4470c43191 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:32:52,191 [197] [DEBUG] [app] Ending request: urn:request:083a7742-065f-48ca-9c73-cb4470c43191 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:083a7742-065f-48ca-9c73-cb4470c43191', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:32:52,191 [199] [DEBUG] [app] Starting request: urn:request:210d6f5c-fa51-4733-8588-c87e076de6e4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:32:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:32:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:52,191 [199] [DEBUG] [app] Ending request: urn:request:210d6f5c-fa51-4733-8588-c87e076de6e4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:210d6f5c-fa51-4733-8588-c87e076de6e4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:32:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:32:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:32:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:32:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:32:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:32:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:32:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:32:52,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:32:52,203 [200] [DEBUG] [app] Ending request: urn:request:64ee8489-cbff-4138-aa70-1dcd1c068437 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:64ee8489-cbff-4138-aa70-1dcd1c068437', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:32:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:32:52,203 [198] [DEBUG] [app] Ending request: urn:request:7391d500-61c0-4dbc-9fff-a8795a13e0d9 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7391d500-61c0-4dbc-9fff-a8795a13e0d9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:32:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:32:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:32:52,203 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:32:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:32:52,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:32:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:32:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:32:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:32:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:02.293757+00:00 (in 9.999483 seconds) chunkcleanupworker stdout | 2023-08-08 06:32:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:02 UTC)" (scheduled at 2023-08-08 06:32:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:32:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:32:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 52, 294632), True, datetime.datetime(2023, 8, 8, 6, 32, 52, 294632), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:32:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:32:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:32:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:02 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:32:52,758 [200] [DEBUG] [app] Starting request: urn:request:264d889c-0189-4f9e-ba9f-66fe64e303f3 (/api/v1/repository/user1_build_test/github/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:32:52,758 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:52,759 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:52,760 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:52,766 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:32:52,766 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:32:52,766 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:52,766 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:32:52,766 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:32:52,767 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:52,767 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:52,767 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:32:52,769 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:32:52,776 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:32:52,777 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:32:52,781 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:32:52,785 [200] [DEBUG] [app] Ending request: urn:request:264d889c-0189-4f9e-ba9f-66fe64e303f3 (/api/v1/repository/user1_build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:264d889c-0189-4f9e-ba9f-66fe64e303f3', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/build/?limit=3', 'path': '/api/v1/repository/user1_build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:32:52,785 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:32:52 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 877 0.029) gunicorn-web stdout | 2023-08-08 06:32:52,786 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:32:52 +0000] "GET /api/v1/repository/user1_build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityscanningnotificationworker stdout | 2023-08-08 06:32:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:32:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:13.097375+00:00 (in 18.812771 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:32:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:54 UTC)" (scheduled at 2023-08-08 06:32:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:32:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:32:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:32:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:32:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:36.579841+00:00 (in 42.185651 seconds) namespacegcworker stdout | 2023-08-08 06:32:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:54 UTC)" (scheduled at 2023-08-08 06:32:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:32:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:32:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 54, 394816), True, datetime.datetime(2023, 8, 8, 6, 32, 54, 394816), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:32:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:32:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:32:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:33:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:32:54,733 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:32:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:32:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:02.178530+00:00 (in 6.806233 seconds) notificationworker stdout | 2023-08-08 06:32:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:05 UTC)" (scheduled at 2023-08-08 06:32:55.371682+00:00) notificationworker stdout | 2023-08-08 06:32:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:32:55,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 32, 55, 372833), True, datetime.datetime(2023, 8, 8, 6, 32, 55, 372833), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:32:55,383 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:32:55,383 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:32:55,383 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:32:55,442 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:32:55,445 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:32:55,447 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:32:55,522 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:32:55,532 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:32:55,532 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:32:55,537 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:32:56,580 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:32:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:32:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:19.687363+00:00 (in 22.796027 seconds) securityworker stdout | 2023-08-08 06:32:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:26 UTC)" (scheduled at 2023-08-08 06:32:56.890741+00:00) securityworker stdout | 2023-08-08 06:32:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:32:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:32:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:32:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:32:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:32:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:32:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:32:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:32:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:32:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:32:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 27, 56, 896876), 1, 2]) securityworker stdout | 2023-08-08 06:32:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 27, 56, 896876), 1, 2]) securityworker stdout | 2023-08-08 06:32:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:32:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:32:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:32:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:32:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:32:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:32:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:32:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:32:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:32:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:26 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:32:56,970 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:32:56,990 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:32:57,840 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:32:58,068 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:32:58,086 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:32:58,197 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:32:58,204 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:32:58,208 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:32:58,576 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:32:58,763 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:32:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:32:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:09.417660+00:00 (in 9.999441 seconds) gcworker stdout | 2023-08-08 06:32:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:09 UTC)" (scheduled at 2023-08-08 06:32:59.417660+00:00) gcworker stdout | 2023-08-08 06:32:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266779418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:32:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:32:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:32:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:33:00,112 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:33:00,263 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:33:01,259 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:33:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:33:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:05.371682+00:00 (in 3.192539 seconds) notificationworker stdout | 2023-08-08 06:33:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:02 UTC)" (scheduled at 2023-08-08 06:33:02.178530+00:00) notificationworker stdout | 2023-08-08 06:33:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:33:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:02 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:33:02,205 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} chunkcleanupworker stdout | 2023-08-08 06:33:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:33:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:12.293757+00:00 (in 9.999544 seconds) chunkcleanupworker stdout | 2023-08-08 06:33:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:12 UTC)" (scheduled at 2023-08-08 06:33:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:33:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:33:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 2, 294539), True, datetime.datetime(2023, 8, 8, 6, 33, 2, 294539), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:33:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:33:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:33:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:12 UTC)" executed successfully servicekey stdout | 2023-08-08 06:33:02,307 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:33:02,712 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:33:03,199 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:33:03,489 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:33:03,786 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:33:03,789 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:33:03,793 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:33:03,797 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:33:03,800 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:33:03,806 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:33:03,812 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:33:03,817 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:33:03,823 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:33:03,950 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} repositorygcworker stdout | 2023-08-08 06:33:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:33:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:09.793658+00:00 (in 5.605604 seconds) repositorygcworker stdout | 2023-08-08 06:33:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:04 UTC)" (scheduled at 2023-08-08 06:33:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:33:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:33:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:33:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:33:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:15.371682+00:00 (in 9.999365 seconds) notificationworker stdout | 2023-08-08 06:33:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:15 UTC)" (scheduled at 2023-08-08 06:33:05.371682+00:00) notificationworker stdout | 2023-08-08 06:33:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:33:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 5, 372828), True, datetime.datetime(2023, 8, 8, 6, 33, 5, 372828), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:33:05,383 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:33:05,383 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:33:05,383 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:15 UTC)" executed successfully builder stdout | 2023-08-08 06:33:05,546 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:33:05,555 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:33:05,556 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:33:05,560 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:33:07,179 [197] [DEBUG] [app] Starting request: urn:request:856514e7-bf50-4b28-91be-b0031d63f450 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:33:07,179 [200] [DEBUG] [app] Starting request: urn:request:78676e4c-1d0a-4665-82ce-105937ad927f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:33:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:33:07,185 [203] [DEBUG] [app] Starting request: urn:request:ac0de1d5-da12-49b8-855b-979b6362f131 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:33:07,185 [209] [DEBUG] [app] Starting request: urn:request:c1ba7ee3-8dc2-4845-964a-a33efad9bd06 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:33:07,185 [203] [DEBUG] [app] Ending request: urn:request:ac0de1d5-da12-49b8-855b-979b6362f131 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ac0de1d5-da12-49b8-855b-979b6362f131', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:33:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:33:07,186 [209] [DEBUG] [app] Ending request: urn:request:c1ba7ee3-8dc2-4845-964a-a33efad9bd06 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c1ba7ee3-8dc2-4845-964a-a33efad9bd06', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:33:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:33:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:33:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:07,191 [198] [DEBUG] [app] Starting request: urn:request:d3d1dc77-56e1-4de2-aebc-311f88c6377f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:07,191 [197] [DEBUG] [app] Starting request: urn:request:48e4bf03-76c3-42aa-ab2b-f6454d42780c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:07,191 [197] [DEBUG] [app] Ending request: urn:request:48e4bf03-76c3-42aa-ab2b-f6454d42780c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:48e4bf03-76c3-42aa-ab2b-f6454d42780c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:33:07,191 [198] [DEBUG] [app] Ending request: urn:request:d3d1dc77-56e1-4de2-aebc-311f88c6377f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d3d1dc77-56e1-4de2-aebc-311f88c6377f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:33:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:33:07,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:33:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:33:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:33:07,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:33:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:33:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:33:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:37.197412+00:00 (in 29.999492 seconds) buildlogsarchiver stdout | 2023-08-08 06:33:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:37 UTC)" (scheduled at 2023-08-08 06:33:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:33:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 33, 7, 198490), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:07,202 [200] [DEBUG] [app] Ending request: urn:request:78676e4c-1d0a-4665-82ce-105937ad927f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:78676e4c-1d0a-4665-82ce-105937ad927f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:33:07,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:33:07,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:33:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:33:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:33:07,203 [197] [DEBUG] [app] Ending request: urn:request:856514e7-bf50-4b28-91be-b0031d63f450 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:856514e7-bf50-4b28-91be-b0031d63f450', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:33:07,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:33:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:33:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:33:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:33:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:33:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:33:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:33:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:33:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:33:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:33:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:19.417660+00:00 (in 9.999436 seconds) gcworker stdout | 2023-08-08 06:33:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:19 UTC)" (scheduled at 2023-08-08 06:33:09.417660+00:00) gcworker stdout | 2023-08-08 06:33:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266789418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:33:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:33:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:33:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:33:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:33:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:04.187429+00:00 (in 54.393202 seconds) repositorygcworker stdout | 2023-08-08 06:33:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:09 UTC)" (scheduled at 2023-08-08 06:33:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:33:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:33:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 9, 794555), True, datetime.datetime(2023, 8, 8, 6, 33, 9, 794555), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:33:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:33:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:33:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:09 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:11,796 [197] [DEBUG] [app] Starting request: urn:request:0ffce97a-3e24-4f4b-80f5-ef1714740657 (/oauth2/gitlab/callback/trigger) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:11,796 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:11,796 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:11,797 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:11,804 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:11,804 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:11,804 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:11,804 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:11,804 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:11,804 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:11,807 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:11,813 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:11,816 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:33:12,089 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "POST /oauth/token?code=3d5f2dd3550f19da235a3bc90b67bea81867c74f88eb97fa8f4c5efffa25daa2&grant_type=authorization_code&redirect_uri=https%3A%2F%2Fquayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com%2Foauth2%2Fgitlab%2Fcallback%2Ftrigger&client_id=e6224cb14edfcc638a26b98e3708b3cd521e79e5600afa1dba37ee297b9fc38e&client_secret=ccbbdfc399411f5a1a684ef5280e5e27954593d237067c84aaa9db50c5aa2169 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:12,090 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:12,093 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:12,095 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "buildtriggerservice" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['gitlab', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:12,096 [197] [DEBUG] [peewee] ('INSERT INTO "repositorybuildtrigger" ("uuid", "service_id", "repository_id", "connected_user_id", "secure_auth_token", "fully_migrated", "config", "pull_robot_id", "enabled", "disabled_datetime", "successive_failure_count", "successive_internal_error_count") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repositorybuildtrigger"."id"', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 2, 4, 3, 'v0$$ggebZXV5j0t840ye6C/dxFCWK6JUd6dn6+kGiCbjbu6fcJXcJsfKRY2sWMx17z11WfNME9bHIqtxhJhfp64yY676quxtDusEc1Tj2/5I0T+0U83r9KHLgnSOO4os', False, '{}', None, True, datetime.datetime(2023, 8, 8, 6, 33, 12, 96136), 0, 0]) gunicorn-web stdout | 2023-08-08 06:33:12,101 [197] [DEBUG] [endpoints.gitlabtrigger] Redirecting to full url: /repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c gunicorn-web stdout | 2023-08-08 06:33:12,102 [197] [DEBUG] [app] Ending request: urn:request:0ffce97a-3e24-4f4b-80f5-ef1714740657 (/oauth2/gitlab/callback/trigger) {'endpoint': 'gitlab.attach_gitlab_build_trigger', 'request_id': 'urn:request:0ffce97a-3e24-4f4b-80f5-ef1714740657', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/oauth2/gitlab/callback/trigger?code=3d5f2dd3550f19da235a3bc90b67bea81867c74f88eb97fa8f4c5efffa25daa2&state=repo%3Auser1_build_test%2Fgithub', 'path': '/oauth2/gitlab/callback/trigger', 'parameters': {'code': '3d5f2dd3550f19da235a3bc90b67bea81867c74f88eb97fa8f4c5efffa25daa2', 'state': 'repo:user1_build_test/github'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:12,103 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:12 +0000] "GET /oauth2/gitlab/callback/trigger?code=3d5f2dd3550f19da235a3bc90b67bea81867c74f88eb97fa8f4c5efffa25daa2&state=repo%3Auser1_build_test%2Fgithub HTTP/2.0" 302 367 "https://gitlab.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.308 845 0.308) gunicorn-web stdout | 2023-08-08 06:33:12,103 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:12 +0000] "GET /oauth2/gitlab/callback/trigger?code=3d5f2dd3550f19da235a3bc90b67bea81867c74f88eb97fa8f4c5efffa25daa2&state=repo%3Auser1_build_test%2Fgithub HTTP/1.0" 302 367 "https://gitlab.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" chunkcleanupworker stdout | 2023-08-08 06:33:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:33:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:22.293757+00:00 (in 9.999429 seconds) chunkcleanupworker stdout | 2023-08-08 06:33:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:22 UTC)" (scheduled at 2023-08-08 06:33:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:33:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:33:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 12, 294727), True, datetime.datetime(2023, 8, 8, 6, 33, 12, 294727), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:33:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:33:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:33:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:22 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:12,564 [200] [DEBUG] [app] Starting request: urn:request:6d9d9919-21aa-4440-b6de-052f8a9f82a2 (/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:12,566 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:12,568 [200] [DEBUG] [app] Ending request: urn:request:6d9d9919-21aa-4440-b6de-052f8a9f82a2 (/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c) {'endpoint': 'web.buildtrigger', 'request_id': 'urn:request:6d9d9919-21aa-4440-b6de-052f8a9f82a2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c', 'path': '/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:12,569 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:12 +0000] "GET /repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c HTTP/1.0" 200 85192 "https://gitlab.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:12 +0000] "GET /repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c HTTP/2.0" 200 13315 "https://gitlab.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.008 434 0.007) securityscanningnotificationworker stdout | 2023-08-08 06:33:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:33:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:54.284093+00:00 (in 41.186136 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:33:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:13 UTC)" (scheduled at 2023-08-08 06:33:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:33:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:33:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 13, 98450), True, datetime.datetime(2023, 8, 8, 6, 33, 13, 98450), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:33:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:33:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:33:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:13 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /static/build/83-quay-frontend.chunk.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 250 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /static/build/135-quay-frontend.chunk.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 79 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /static/build/18-quay-frontend.chunk.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 79 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /static/build/112-quay-frontend.chunk.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 80 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /static/build/181-quay-frontend.chunk.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 80 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /static/build/152-quay-frontend.chunk.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 79 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /static/build/489e5e00e1a9a80f88d55281894a87ff.woff HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 155 -) gunicorn-web stdout | 2023-08-08 06:33:13,419 [198] [DEBUG] [app] Starting request: urn:request:cfb4e39f-a05a-4601-ad30-75cbbbab30bc (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:13,419 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,419 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,420 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,427 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:13,427 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:13,427 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,427 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,427 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,427 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,428 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,429 [198] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'user1']) gunicorn-web stdout | 2023-08-08 06:33:13,432 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,432 [198] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 3]) gunicorn-web stdout | 2023-08-08 06:33:13,434 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [3]) gunicorn-web stdout | 2023-08-08 06:33:13,435 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,437 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:33:13,438 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,441 [198] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691476393439, 0, 3, None]) gunicorn-web stdout | 2023-08-08 06:33:13,444 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,446 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:33:13,447 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,449 [198] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691476393448, 0, 3]) gunicorn-web stdout | 2023-08-08 06:33:13,450 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,450 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,451 [199] [DEBUG] [app] Starting request: urn:request:078e668e-8d1c-4d6f-91d7-0803718c0df8 (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:13,451 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,451 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,451 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:33:13,452 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,453 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,454 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,454 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,454 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,454 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,455 [198] [DEBUG] [app] Ending request: urn:request:cfb4e39f-a05a-4601-ad30-75cbbbab30bc (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:cfb4e39f-a05a-4601-ad30-75cbbbab30bc', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:13,456 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/user/ HTTP/2.0" 200 918 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.039 167 0.039) gunicorn-web stdout | 2023-08-08 06:33:13,456 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/user/ HTTP/1.0" 200 918 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:33:13,458 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:13,458 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:13,458 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,459 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 06:33:13,461 [199] [DEBUG] [app] Ending request: urn:request:078e668e-8d1c-4d6f-91d7-0803718c0df8 (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:078e668e-8d1c-4d6f-91d7-0803718c0df8', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:13,461 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:33:13,462 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 47 0.012) gunicorn-web stdout | 2023-08-08 06:33:13,516 [198] [DEBUG] [app] Starting request: urn:request:d1c20e7d-e9c5-408e-889c-508ed5fea617 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:13,516 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,516 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,516 [200] [DEBUG] [app] Starting request: urn:request:f1ea1aaa-a25a-408b-aa09-8d5ecb4e67d9 (/api/v1/repository/user1_build_test/github) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:13,517 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,517 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,517 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,518 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,524 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:13,524 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:13,524 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,524 [198] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:13,524 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,524 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,524 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,524 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,524 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:13,525 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:13,525 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,525 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:13,525 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,525 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,525 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,525 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,527 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:13,529 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:13,533 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,534 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,535 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,536 [200] [DEBUG] [endpoints.api.repository] Get repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:13,537 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,537 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,539 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [4, 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,540 [198] [DEBUG] [app] Ending request: urn:request:d1c20e7d-e9c5-408e-889c-508ed5fea617 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:d1c20e7d-e9c5-408e-889c-508ed5fea617', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:13,541 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c HTTP/2.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.027 97 0.027) gunicorn-web stdout | 2023-08-08 06:33:13,541 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c HTTP/1.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:33:13,541 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [4, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:33:13,542 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,542 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,543 [200] [DEBUG] [app] Ending request: urn:request:f1ea1aaa-a25a-408b-aa09-8d5ecb4e67d9 (/api/v1/repository/user1_build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:f1ea1aaa-a25a-408b-aa09-8d5ecb4e67d9', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github?includeTags=false', 'path': '/api/v1/repository/user1_build_test/github', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:13,545 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/repository/user1_build_test/github?includeTags=false HTTP/2.0" 200 345 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.031 79 0.031) gunicorn-web stdout | 2023-08-08 06:33:13,545 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/repository/user1_build_test/github?includeTags=false HTTP/1.0" 200 345 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:33:13,794 [199] [DEBUG] [app] Starting request: urn:request:4aa0244c-b784-474c-8db7-79fe962c0c7b (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:13,794 [200] [DEBUG] [app] Starting request: urn:request:59c2ceb0-e20f-470e-9981-949d237aeac7 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:13,794 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,795 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,795 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,795 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,795 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,795 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,802 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:13,802 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:13,802 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,803 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:13,803 [200] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:33:13,803 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,803 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,803 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:13,803 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,803 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,803 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,803 [199] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:33:13,803 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,803 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,803 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,803 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,804 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 45, 13, 803701), datetime.datetime(2023, 8, 8, 6, 33, 13, 803752)]) gunicorn-web stdout | 2023-08-08 06:33:13,804 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:33:13,805 [199] [DEBUG] [app] Ending request: urn:request:4aa0244c-b784-474c-8db7-79fe962c0c7b (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:4aa0244c-b784-474c-8db7-79fe962c0c7b', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:13,806 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 62 0.014) gunicorn-web stdout | 2023-08-08 06:33:13,806 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:33:13,809 [200] [DEBUG] [app] Ending request: urn:request:59c2ceb0-e20f-470e-9981-949d237aeac7 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:59c2ceb0-e20f-470e-9981-949d237aeac7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:13,810 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 397 0.017) gunicorn-web stdout | 2023-08-08 06:33:13,810 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:13 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:33:13,897 [200] [DEBUG] [app] Starting request: urn:request:e4068359-7ebc-4668-a832-ac4cd9fb1dd2 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/namespaces) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:13,897 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,897 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,898 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,905 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:13,905 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:13,905 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,905 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:13,905 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,905 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,905 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,905 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,908 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:13,914 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:13,915 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,918 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,921 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:13,922 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:13,956 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:33:14,162 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:14 +0000] "GET /static/img/quay_favicon.png HTTP/2.0" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 81 -) gunicorn-web stdout | 2023-08-08 06:33:14,432 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/namespaces?page=1&per_page=20 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:14,435 [200] [DEBUG] [app] Ending request: urn:request:e4068359-7ebc-4668-a832-ac4cd9fb1dd2 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/namespaces) {'endpoint': 'api.buildtriggersourcenamespaces', 'request_id': 'urn:request:e4068359-7ebc-4668-a832-ac4cd9fb1dd2', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/namespaces', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/namespaces', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:14,435 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:33:14,436 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:14 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/namespaces HTTP/1.0" 200 365 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:14 +0000] "GET /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/namespaces HTTP/2.0" 200 365 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.541 105 0.540) notificationworker stdout | 2023-08-08 06:33:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:33:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:25.371682+00:00 (in 9.999443 seconds) notificationworker stdout | 2023-08-08 06:33:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:25 UTC)" (scheduled at 2023-08-08 06:33:15.371682+00:00) notificationworker stdout | 2023-08-08 06:33:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:33:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 15, 372840), True, datetime.datetime(2023, 8, 8, 6, 33, 15, 372840), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:33:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:33:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:33:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:25 UTC)" executed successfully builder stdout | 2023-08-08 06:33:15,571 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:33:15,579 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:33:15,579 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:33:15,584 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:33:16,402 [199] [DEBUG] [app] Starting request: urn:request:58f3bedb-6d0e-4b5c-a154-794360d00329 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/sources) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:16,402 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:16,402 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:16,403 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:16,410 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:16,410 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:16,410 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:16,410 [199] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:16,410 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:16,410 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:16,410 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:16,410 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:16,413 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:16,419 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:16,420 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:16,423 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:16,426 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:16,429 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:16,430 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:16,462 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:33:16,666 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:16,878 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/namespaces/52807017 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:17,239 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/groups/52807017/projects?page=1&per_page=20 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:17,242 [199] [DEBUG] [app] Ending request: urn:request:58f3bedb-6d0e-4b5c-a154-794360d00329 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/sources) {'endpoint': 'api.buildtriggersources', 'request_id': 'urn:request:58f3bedb-6d0e-4b5c-a154-794360d00329', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/sources', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/sources', 'parameters': {}, 'json_body': {'namespace': '52807017'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:17,243 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:17 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/sources HTTP/2.0" 200 503 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.844 566 0.843) gunicorn-web stdout | 2023-08-08 06:33:17,244 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:17 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/sources HTTP/1.0" 200 503 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:33:19,351 [199] [DEBUG] [app] Starting request: urn:request:31ac639a-a86c-4d02-a595-5e9d87417bac (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/subdir) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:19,351 [200] [DEBUG] [app] Starting request: urn:request:f25647f5-7949-4291-881c-0d18828b81ce (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/fields/refs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:19,351 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,351 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,351 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,351 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,352 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,352 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,359 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:19,359 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:19,359 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:19,359 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,359 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:19,359 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,360 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:19,360 [199] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:19,360 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,360 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,360 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:19,360 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:19,360 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:19,360 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:19,360 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:19,360 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:19,362 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:19,362 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:19,368 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:19,369 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:19,370 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,370 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,372 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,372 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,375 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,375 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,377 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,378 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:19,379 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:19,379 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:33:19,381 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gcworker stdout | 2023-08-08 06:33:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:33:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:29.417660+00:00 (in 9.999490 seconds) gcworker stdout | 2023-08-08 06:33:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:29 UTC)" (scheduled at 2023-08-08 06:33:19.417660+00:00) gcworker stdout | 2023-08-08 06:33:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266799418, 1209600, True, 3, 500, 1, 0]) notificationworker stdout | 2023-08-08 06:33:19,424 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} gcworker stdout | 2023-08-08 06:33:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:33:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:33:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:29 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:19,566 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:19,600 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:33:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:33:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:26.890741+00:00 (in 7.202849 seconds) securityworker stdout | 2023-08-08 06:33:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:49 UTC)" (scheduled at 2023-08-08 06:33:19.687363+00:00) securityworker stdout | 2023-08-08 06:33:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:33:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:33:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:33:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:33:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:33:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:33:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 28, 19, 691784), 1, 2]) securityworker stdout | 2023-08-08 06:33:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 28, 19, 691784), 1, 2]) securityworker stdout | 2023-08-08 06:33:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:33:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:49 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:19,878 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:19,878 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:20,125 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:20,126 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:20,128 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:33:20,336 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:20,377 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/tree?recursive=False&ref=main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:20,378 [199] [DEBUG] [app] Ending request: urn:request:31ac639a-a86c-4d02-a595-5e9d87417bac (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/subdir) {'endpoint': 'api.buildtriggersubdirs', 'request_id': 'urn:request:31ac639a-a86c-4d02-a595-5e9d87417bac', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/subdir', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/subdir', 'parameters': {}, 'json_body': {'build_source': 'h4051/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:20,379 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:20 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/subdir HTTP/2.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.030 145 1.031) gunicorn-web stdout | 2023-08-08 06:33:20,379 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:20 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/subdir HTTP/1.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" logrotateworker stdout | 2023-08-08 06:33:20,502 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:33:20,616 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:20,832 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/tags HTTP/1.1" 200 2 gunicorn-web stdout | 2023-08-08 06:33:20,833 [200] [DEBUG] [app] Ending request: urn:request:f25647f5-7949-4291-881c-0d18828b81ce (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/fields/refs) {'endpoint': 'api.buildtriggerfieldvalues', 'request_id': 'urn:request:f25647f5-7949-4291-881c-0d18828b81ce', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/fields/refs', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/fields/refs', 'parameters': {}, 'json_body': {'build_source': 'h4051/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:20,834 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:20 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/fields/refs HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.486 696 1.486) gunicorn-web stdout | 2023-08-08 06:33:20,835 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:20 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/fields/refs HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" manifestbackfillworker stdout | 2023-08-08 06:33:21,646 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:33:22,091 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:33:22,179 [197] [DEBUG] [app] Starting request: urn:request:28c951ee-c4da-4cbb-8d95-56ad15dee0ca (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:33:22,180 [200] [DEBUG] [app] Starting request: urn:request:b02a7177-6ba3-40b5-92c4-9883a8e9b761 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:33:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:22,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:22,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:33:22,185 [205] [DEBUG] [app] Starting request: urn:request:06b8d121-b4f2-43ca-b49b-72f73a40eada (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:33:22,185 [205] [DEBUG] [app] Ending request: urn:request:06b8d121-b4f2-43ca-b49b-72f73a40eada (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:06b8d121-b4f2-43ca-b49b-72f73a40eada', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:33:22,186 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:33:22,186 [208] [DEBUG] [app] Starting request: urn:request:d26b06dd-bdd7-43c0-9f5c-fcfef78d9186 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:33:22,186 [208] [DEBUG] [app] Ending request: urn:request:d26b06dd-bdd7-43c0-9f5c-fcfef78d9186 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d26b06dd-bdd7-43c0-9f5c-fcfef78d9186', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:33:22,187 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:33:22,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:22,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:22,191 [198] [DEBUG] [app] Starting request: urn:request:cd49b9d0-3406-48a0-bc6e-7eb03b62053d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:22,192 [198] [DEBUG] [app] Ending request: urn:request:cd49b9d0-3406-48a0-bc6e-7eb03b62053d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:cd49b9d0-3406-48a0-bc6e-7eb03b62053d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:33:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:33:22,192 [199] [DEBUG] [app] Starting request: urn:request:ceb5b3a7-4fdd-4939-a686-9306201b29ec (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:22,193 [199] [DEBUG] [app] Ending request: urn:request:ceb5b3a7-4fdd-4939-a686-9306201b29ec (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ceb5b3a7-4fdd-4939-a686-9306201b29ec', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:33:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:33:22,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:33:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:33:22,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:33:22,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:33:22,203 [197] [DEBUG] [app] Ending request: urn:request:28c951ee-c4da-4cbb-8d95-56ad15dee0ca (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:28c951ee-c4da-4cbb-8d95-56ad15dee0ca', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:33:22,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:33:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:33:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:33:22,204 [200] [DEBUG] [app] Ending request: urn:request:b02a7177-6ba3-40b5-92c4-9883a8e9b761 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b02a7177-6ba3-40b5-92c4-9883a8e9b761', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:33:22,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:33:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:33:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:33:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:33:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:33:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:33:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:28.874906+00:00 (in 6.580628 seconds) chunkcleanupworker stdout | 2023-08-08 06:33:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:32 UTC)" (scheduled at 2023-08-08 06:33:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:33:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:33:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 22, 294640), True, datetime.datetime(2023, 8, 8, 6, 33, 22, 294640), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:33:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:33:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:33:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:33:24,770 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:33:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:33:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:35.371682+00:00 (in 9.999414 seconds) notificationworker stdout | 2023-08-08 06:33:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:35 UTC)" (scheduled at 2023-08-08 06:33:25.371682+00:00) notificationworker stdout | 2023-08-08 06:33:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:33:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 25, 372818), True, datetime.datetime(2023, 8, 8, 6, 33, 25, 372818), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:33:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:33:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:33:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:35 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:33:25,451 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:33:25,454 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:33:25,458 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} exportactionlogsworker stdout | 2023-08-08 06:33:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:33:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:33.514094+00:00 (in 7.999195 seconds) exportactionlogsworker stdout | 2023-08-08 06:33:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:25 UTC)" (scheduled at 2023-08-08 06:33:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:33:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:33:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:25 UTC)" executed successfully builder stdout | 2023-08-08 06:33:25,594 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:33:25,602 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:33:25,603 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:33:25,607 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:33:26,618 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:33:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:33:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:49.687363+00:00 (in 22.796139 seconds) securityworker stdout | 2023-08-08 06:33:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:56 UTC)" (scheduled at 2023-08-08 06:33:26.890741+00:00) securityworker stdout | 2023-08-08 06:33:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:33:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:33:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:33:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:33:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:33:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:33:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:33:26,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:33:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:33:26,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:33:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 28, 26, 896165), 1, 2]) securityworker stdout | 2023-08-08 06:33:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 28, 26, 896165), 1, 2]) securityworker stdout | 2023-08-08 06:33:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:33:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:33:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:33:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:33:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:33:26,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:33:26,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:33:26,908 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:33:56 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:33:27,007 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:33:27,027 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:33:27,848 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} teamsyncworker stdout | 2023-08-08 06:33:28,095 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:33:28,099 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:33:28,211 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:33:28,217 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:33:28,220 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:33:28,611 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:33:28,800 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:33:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:33:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:32.293757+00:00 (in 3.418363 seconds) chunkcleanupworker stdout | 2023-08-08 06:33:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:28 UTC)" (scheduled at 2023-08-08 06:33:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:33:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:33:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:28 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:28,978 [198] [DEBUG] [app] Starting request: urn:request:7e36022c-82d0-4f7f-a490-936c1c79293d (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:28,979 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:28,979 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:28,979 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:28,986 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:28,987 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:28,987 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:28,987 [198] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:28,987 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:28,987 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:28,987 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:28,987 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:28,989 [198] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:28,996 [198] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:28,997 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:28,999 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:29,002 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:29,005 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:29,006 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:33:29,008 [198] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:33:29,008 [198] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:33:29,008 [198] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:33:29,040 [198] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:33:29,230 [198] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gcworker stdout | 2023-08-08 06:33:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:33:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:39.417660+00:00 (in 9.999502 seconds) gcworker stdout | 2023-08-08 06:33:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:39 UTC)" (scheduled at 2023-08-08 06:33:29.417660+00:00) gcworker stdout | 2023-08-08 06:33:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266809418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:33:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:33:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:33:29,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:39 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:29,503 [198] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:29,505 [198] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:33:29,815 [198] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:30,086 [198] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None buildlogsarchiver stdout | 2023-08-08 06:33:30,134 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:33:30,300 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gunicorn-web stdout | 2023-08-08 06:33:30,305 [198] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:30,539 [198] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/files/Dockerfile?ref=main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:30,541 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:33:30,544 [198] [DEBUG] [app] Ending request: urn:request:7e36022c-82d0-4f7f-a490-936c1c79293d (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:7e36022c-82d0-4f7f-a490-936c1c79293d', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'h4051/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': None}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:30,544 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:30 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze HTTP/2.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.568 1121 1.568) gunicorn-web stdout | 2023-08-08 06:33:30,545 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:30 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze HTTP/1.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityworker stdout | 2023-08-08 06:33:31,296 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:33:32,214 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} chunkcleanupworker stdout | 2023-08-08 06:33:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:33:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:42.293757+00:00 (in 9.999480 seconds) chunkcleanupworker stdout | 2023-08-08 06:33:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:42 UTC)" (scheduled at 2023-08-08 06:33:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:33:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:33:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 32, 294630), True, datetime.datetime(2023, 8, 8, 6, 33, 32, 294630), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:33:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:33:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:33:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:42 UTC)" executed successfully servicekey stdout | 2023-08-08 06:33:32,318 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} gunicorn-web stdout | 2023-08-08 06:33:32,659 [200] [DEBUG] [app] Starting request: urn:request:f09a05b1-b0de-46ba-8da6-79e3caf8035f (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:32,659 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:32,659 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:32,660 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:32,667 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:32,667 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:32,667 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:32,667 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:32,667 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:32,667 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:32,667 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:32,667 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:32,670 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:32,676 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:32,677 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:32,679 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:32,682 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:32,685 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:32,686 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:33:32,688 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:33:32,688 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:33:32,688 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:33:32,690 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 globalpromstats stdout | 2023-08-08 06:33:32,721 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} gunicorn-web stdout | 2023-08-08 06:33:32,876 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:33,133 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:33,136 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 securityscanningnotificationworker stdout | 2023-08-08 06:33:33,234 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-web stdout | 2023-08-08 06:33:33,342 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None exportactionlogsworker stdout | 2023-08-08 06:33:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:33:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:47.398551+00:00 (in 13.883857 seconds) exportactionlogsworker stdout | 2023-08-08 06:33:33,514 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:38:33 UTC)" (scheduled at 2023-08-08 06:33:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:33:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 33, 33, 515099), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:33:33,522 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 33, 33, 515099), True, datetime.datetime(2023, 8, 8, 6, 33, 33, 515099), 0, 'exportactionlogs/%']) namespacegcworker stdout | 2023-08-08 06:33:33,523 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} exportactionlogsworker stdout | 2023-08-08 06:33:33,523 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 33, 33, 515099), True, datetime.datetime(2023, 8, 8, 6, 33, 33, 515099), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 33, 33, 515099), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:33:33,525 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:33:33,525 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:38:33 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:33,596 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:33,775 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-registry stdout | 2023-08-08 06:33:33,800 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:33:33,806 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:33:33,810 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:33:33,815 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:33:33,820 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:33:33,825 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:33:33,829 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:33:33,834 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:33:33,840 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:33:33,977 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} gunicorn-web stdout | 2023-08-08 06:33:34,021 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/files/Dockerfile?ref=main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:34,023 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:33:34,025 [200] [DEBUG] [app] Ending request: urn:request:f09a05b1-b0de-46ba-8da6-79e3caf8035f (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:f09a05b1-b0de-46ba-8da6-79e3caf8035f', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'h4051/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:34,026 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:34 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze HTTP/2.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.369 549 1.370) gunicorn-web stdout | 2023-08-08 06:33:34,026 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:34 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/analyze HTTP/1.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:33:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:33:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:45.371682+00:00 (in 9.999425 seconds) notificationworker stdout | 2023-08-08 06:33:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:45 UTC)" (scheduled at 2023-08-08 06:33:35.371682+00:00) notificationworker stdout | 2023-08-08 06:33:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:33:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 35, 372777), True, datetime.datetime(2023, 8, 8, 6, 33, 35, 372777), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:33:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:33:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:33:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:45 UTC)" executed successfully builder stdout | 2023-08-08 06:33:35,617 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:33:35,626 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:33:35,626 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:33:35,631 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:33:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:33:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:54.393627+00:00 (in 17.813185 seconds) namespacegcworker stdout | 2023-08-08 06:33:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:36 UTC)" (scheduled at 2023-08-08 06:33:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:33:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:33:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:37,031 [197] [DEBUG] [app] Starting request: urn:request:309a1b3b-8780-486b-85e8-92dc9b08a627 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:33:37,031 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:37,031 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:37,032 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:37,038 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:33:37,038 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:33:37,038 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:37,038 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:33:37,039 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:37,039 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:37,039 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:37,039 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:33:37,041 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:33:37,047 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:33:37,048 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:37,051 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:37,054 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:37,057 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:37,058 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:33:37,059 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['write', 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:37,061 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:37,063 [197] [DEBUG] [peewee] ('INSERT INTO "accesstoken" ("friendly_name", "token_name", "token_code", "repository_id", "created", "role_id", "temporary") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "accesstoken"."id"', ['Build Trigger: gitlab', 'O2RDITIP3US7ULFVXJM2M0MDGZ4AZDP1', 'v0$$tm0bxuvIBLGtPwI7O19n+kEq77WlPCZ6tDoVieKaoVvgmc9Uf0Pod1ZWqsUb09dPNispD7lx2dwUtoPZ9g==', 4, datetime.datetime(2023, 8, 8, 6, 33, 37, 62723), 2, False]) gunicorn-web stdout | 2023-08-08 06:33:37,100 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:33:37,179 [199] [DEBUG] [app] Starting request: urn:request:8eed195e-542f-4963-8369-5283877357ff (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:33:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:37,181 [200] [DEBUG] [app] Starting request: urn:request:27236a34-2e8f-4c67-8cb0-83398b3534ee (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:33:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:37,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:33:37,184 [210] [DEBUG] [app] Starting request: urn:request:be399731-1f41-4428-9fbd-3058da3b15b2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:33:37,185 [210] [DEBUG] [app] Ending request: urn:request:be399731-1f41-4428-9fbd-3058da3b15b2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:be399731-1f41-4428-9fbd-3058da3b15b2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:33:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:33:37,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:33:37,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:33:37,186 [203] [DEBUG] [app] Starting request: urn:request:4a53c459-38e6-4ce0-b45a-5761fdca2a73 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:33:37,186 [203] [DEBUG] [app] Ending request: urn:request:4a53c459-38e6-4ce0-b45a-5761fdca2a73 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4a53c459-38e6-4ce0-b45a-5761fdca2a73', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:33:37,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:33:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:37,191 [197] [DEBUG] [app] Starting request: urn:request:629b21e6-b6b3-4ba2-ad29-02e50fa66805 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:37,191 [199] [DEBUG] [app] Starting request: urn:request:b6132a2a-33f0-4ae5-9a71-71652d7891e4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:37,192 [197] [DEBUG] [app] Ending request: urn:request:629b21e6-b6b3-4ba2-ad29-02e50fa66805 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:629b21e6-b6b3-4ba2-ad29-02e50fa66805', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:33:37,192 [199] [DEBUG] [app] Ending request: urn:request:b6132a2a-33f0-4ae5-9a71-71652d7891e4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b6132a2a-33f0-4ae5-9a71-71652d7891e4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:33:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:33:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:33:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:37,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:33:37,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:33:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:33:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:33:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:07.197412+00:00 (in 29.999487 seconds) gunicorn-web stdout | 2023-08-08 06:33:37,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:33:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:07 UTC)" (scheduled at 2023-08-08 06:33:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:33:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 33, 37, 198337), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:33:37,203 [199] [DEBUG] [app] Ending request: urn:request:8eed195e-542f-4963-8369-5283877357ff (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8eed195e-542f-4963-8369-5283877357ff', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:33:37,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:33:37,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:33:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:33:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:33:37,204 [200] [DEBUG] [app] Ending request: urn:request:27236a34-2e8f-4c67-8cb0-83398b3534ee (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:27236a34-2e8f-4c67-8cb0-83398b3534ee', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:33:37,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:33:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:33:37,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:33:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:33:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:33:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:33:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:33:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:33:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:33:37,322 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:37,634 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:33:37,932 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "POST /api/v4/projects/35911436/deploy_keys HTTP/1.1" 201 673 gunicorn-web stdout | 2023-08-08 06:33:38,223 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "POST /api/v4/projects/35911436/hooks HTTP/1.1" 422 29 gunicorn-web stdout | 2023-08-08 06:33:38,224 [197] [ERROR] [buildtrigger.gitlabhandler] GitLab API error. Please contact support. gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 267, in wrapped_f gunicorn-web stdout | return f(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/mixins.py", line 211, in create gunicorn-web stdout | server_data = self.gitlab.http_post(path, post_data=data, files=files, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 680, in http_post gunicorn-web stdout | result = self.http_request( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 573, in http_request gunicorn-web stdout | raise GitlabHttpError( gunicorn-web stdout | gitlab.exceptions.GitlabHttpError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 100, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 270, in activate gunicorn-web stdout | hook = gl_project.hooks.create( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 269, in wrapped_f gunicorn-web stdout | raise error(e.error_message, e.response_code, e.response_body) gunicorn-web stdout | gitlab.exceptions.GitlabCreateError: 422: Invalid url given gunicorn-web stdout | 2023-08-08 06:33:38,225 [197] [ERROR] [app] Exception on /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate [POST] gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 267, in wrapped_f gunicorn-web stdout | return f(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/mixins.py", line 211, in create gunicorn-web stdout | server_data = self.gitlab.http_post(path, post_data=data, files=files, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 680, in http_post gunicorn-web stdout | result = self.http_request( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 573, in http_request gunicorn-web stdout | raise GitlabHttpError( gunicorn-web stdout | gitlab.exceptions.GitlabHttpError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 100, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 270, in activate gunicorn-web stdout | hook = gl_project.hooks.create( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 269, in wrapped_f gunicorn-web stdout | raise error(e.error_message, e.response_code, e.response_body) gunicorn-web stdout | gitlab.exceptions.GitlabCreateError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/app.py", line 1949, in full_dispatch_request gunicorn-web stdout | rv = self.dispatch_request() gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/app.py", line 1935, in dispatch_request gunicorn-web stdout | return self.view_functions[rule.endpoint](**req.view_args) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 256, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/auth/decorators.py", line 65, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/util/request.py", line 50, in wrapper gunicorn-web stdout | return decorator(func)(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/utils/cors.py", line 35, in wrapped_function gunicorn-web stdout | resp = make_response(f(*args, **kwargs)) gunicorn-web stdout | File "/quay-registry/endpoints/csrf.py", line 73, in wrapper gunicorn-web stdout | resp = func(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/__init__.py", line 467, in wrapper gunicorn-web stdout | resp = resource(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/views.py", line 89, in view gunicorn-web stdout | return self.dispatch_request(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/__init__.py", line 582, in dispatch_request gunicorn-web stdout | resp = meth(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 189, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 247, in wrapper gunicorn-web stdout | return func(namespace, repository, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 164, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 342, in wrapped gunicorn-web stdout | return func(self, namespace, repository, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 296, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 308, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 283, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 494, in wrapped gunicorn-web stdout | return func(self, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/trigger.py", line 323, in post gunicorn-web stdout | final_config, private_config = handler.activate(authed_url) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 108, in wrapper gunicorn-web stdout | raise ExternalServiceError(msg) gunicorn-web stdout | endpoints.exception.ExternalServiceError: 520 Unknown Error: GitLab API error. Please contact support. gunicorn-web stdout | 2023-08-08 06:33:38,227 [197] [DEBUG] [app] Ending request: urn:request:309a1b3b-8780-486b-85e8-92dc9b08a627 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate) {'endpoint': 'api.buildtriggeractivate', 'request_id': 'urn:request:309a1b3b-8780-486b-85e8-92dc9b08a627', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate', 'parameters': {}, 'json_body': {'config': {'build_source': 'h4051/quay_build_test', 'dockerfile_path': '/Dockerfile', 'context': '/', 'default_tag_from_ref': True, 'latest_for_default_branch': True, 'tag_templates': [], 'credentials': [{'name': 'SSH Public Key', 'value': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCqZX5EPaJXP0SieDYNZ3laqmSqipw/OUDB5qtGFaY/6CvObJz0A4kSuzUD+H06BkMmeg+b7KxTtHg8NPACV7iDybUwzKqkyoLHwm0D89uFpowWy62eZg8m1YSyUHp6n3erCls5pxpdnaVkgHi0sjmSe3Ax7l46LHU9xlelEBpZRsoEvGzYt+8DACV1EllulGFLPji8mQ/a75dXEjJ3dfzsufPfnI6Jl3AXGjRTcDq306jNhsDLSU33Iqndcsbhj5omp5QmYYk/LB0kaJNwHbzJKBDJmhfm0CtUdk+GZUVRPp+eC5qj4yXAGOF8I+1cupuoJPHtGAR0o9EcZ80ix4iT'}], 'key_id': 12098882}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:33:38,228 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:33:38,228 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:33:38 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate HTTP/1.0" 520 348 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:33:38 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate HTTP/2.0" 520 348 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.199 630 1.200) gcworker stdout | 2023-08-08 06:33:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:33:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:49.417660+00:00 (in 9.999394 seconds) gcworker stdout | 2023-08-08 06:33:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:49 UTC)" (scheduled at 2023-08-08 06:33:39.417660+00:00) gcworker stdout | 2023-08-08 06:33:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266819418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:33:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:33:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:33:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:33:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:33:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:52.293757+00:00 (in 9.999489 seconds) chunkcleanupworker stdout | 2023-08-08 06:33:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:52 UTC)" (scheduled at 2023-08-08 06:33:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:33:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:33:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 42, 294987), True, datetime.datetime(2023, 8, 8, 6, 33, 42, 294987), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:33:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:33:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:33:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:33:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:33:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:49.178248+00:00 (in 3.806092 seconds) notificationworker stdout | 2023-08-08 06:33:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:55 UTC)" (scheduled at 2023-08-08 06:33:45.371682+00:00) notificationworker stdout | 2023-08-08 06:33:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:33:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 45, 373241), True, datetime.datetime(2023, 8, 8, 6, 33, 45, 373241), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:33:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:33:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:33:45,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:55 UTC)" executed successfully builder stdout | 2023-08-08 06:33:45,634 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:33:45,645 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:33:45,645 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:33:45,649 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:33:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:33:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:25.514341+00:00 (in 38.115191 seconds) exportactionlogsworker stdout | 2023-08-08 06:33:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:47 UTC)" (scheduled at 2023-08-08 06:33:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:33:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:33:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 47, 399653), True, datetime.datetime(2023, 8, 8, 6, 33, 47, 399653), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:33:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:33:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:33:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:33:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:33:49,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:55.371682+00:00 (in 6.192733 seconds) notificationworker stdout | 2023-08-08 06:33:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:38:49 UTC)" (scheduled at 2023-08-08 06:33:49.178248+00:00) notificationworker stdout | 2023-08-08 06:33:49,180 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 33, 49, 179656), 'notification/%']) notificationworker stdout | 2023-08-08 06:33:49,187 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 33, 49, 179656), True, datetime.datetime(2023, 8, 8, 6, 33, 49, 179656), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:33:49,189 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 33, 49, 179656), True, datetime.datetime(2023, 8, 8, 6, 33, 49, 179656), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 33, 49, 179656), 'notification/%']) notificationworker stdout | 2023-08-08 06:33:49,190 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:33:49,191 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:38:49 UTC)" executed successfully gcworker stdout | 2023-08-08 06:33:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:33:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:59.417660+00:00 (in 9.999476 seconds) gcworker stdout | 2023-08-08 06:33:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:59 UTC)" (scheduled at 2023-08-08 06:33:49.417660+00:00) gcworker stdout | 2023-08-08 06:33:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266829418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:33:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:33:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:33:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:33:59 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:33:49,459 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} securityworker stdout | 2023-08-08 06:33:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:33:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:33:56.890741+00:00 (in 7.202864 seconds) securityworker stdout | 2023-08-08 06:33:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:19 UTC)" (scheduled at 2023-08-08 06:33:49.687363+00:00) securityworker stdout | 2023-08-08 06:33:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:33:49,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:33:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:33:49,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:33:49,699 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:33:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:33:50,523 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:33:51,670 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:33:52,109 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:33:52,179 [198] [DEBUG] [app] Starting request: urn:request:1db2d31e-8db2-4a2c-8c8d-20c7feaf14bb (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:33:52,179 [200] [DEBUG] [app] Starting request: urn:request:c361a91e-2f8e-4080-b492-b8b73892ae14 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:33:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:33:52,184 [203] [DEBUG] [app] Starting request: urn:request:62ca28a0-16f8-4cfa-9f06-c22385389594 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:33:52,185 [203] [DEBUG] [app] Ending request: urn:request:62ca28a0-16f8-4cfa-9f06-c22385389594 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:62ca28a0-16f8-4cfa-9f06-c22385389594', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:33:52,185 [209] [DEBUG] [app] Starting request: urn:request:fd76adbb-e23f-4662-aae0-590ed83e5a98 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-registry stdout | 2023-08-08 06:33:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:33:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:33:52,186 [209] [DEBUG] [app] Ending request: urn:request:fd76adbb-e23f-4662-aae0-590ed83e5a98 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fd76adbb-e23f-4662-aae0-590ed83e5a98', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:33:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:33:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:52,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:33:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:52,190 [198] [DEBUG] [app] Starting request: urn:request:7ce58627-b17b-4f4d-b81a-9db6229af791 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:52,191 [198] [DEBUG] [app] Ending request: urn:request:7ce58627-b17b-4f4d-b81a-9db6229af791 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7ce58627-b17b-4f4d-b81a-9db6229af791', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:33:52,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:33:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:52,192 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:33:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:33:52,193 [197] [DEBUG] [app] Starting request: urn:request:b70d4f30-5c81-4cab-bda5-f507cb10279b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:33:52,193 [197] [DEBUG] [app] Ending request: urn:request:b70d4f30-5c81-4cab-bda5-f507cb10279b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b70d4f30-5c81-4cab-bda5-f507cb10279b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:33:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:33:52,194 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:33:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:33:52,194 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:33:52,194 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:33:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:33:52,199 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:33:52,203 [200] [DEBUG] [app] Ending request: urn:request:c361a91e-2f8e-4080-b492-b8b73892ae14 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c361a91e-2f8e-4080-b492-b8b73892ae14', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:33:52,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:33:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:33:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:33:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:33:52,204 [198] [DEBUG] [app] Ending request: urn:request:1db2d31e-8db2-4a2c-8c8d-20c7feaf14bb (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1db2d31e-8db2-4a2c-8c8d-20c7feaf14bb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:33:52,204 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:33:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:33:52,205 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:33:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:33:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:33:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:02.293757+00:00 (in 9.999503 seconds) chunkcleanupworker stdout | 2023-08-08 06:33:52,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:02 UTC)" (scheduled at 2023-08-08 06:33:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:33:52,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:33:52,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 52, 295269), True, datetime.datetime(2023, 8, 8, 6, 33, 52, 295269), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:33:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:33:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:33:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:33:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:33:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:13.097375+00:00 (in 18.812685 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:33:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:54 UTC)" (scheduled at 2023-08-08 06:33:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:33:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:33:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:33:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:33:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:36.579841+00:00 (in 42.185653 seconds) namespacegcworker stdout | 2023-08-08 06:33:54,395 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:54 UTC)" (scheduled at 2023-08-08 06:33:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:33:54,395 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:33:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 54, 395399), True, datetime.datetime(2023, 8, 8, 6, 33, 54, 395399), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:33:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:33:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:33:54,404 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:34:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:33:54,778 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:33:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:33:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:02.178530+00:00 (in 6.806224 seconds) notificationworker stdout | 2023-08-08 06:33:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:05 UTC)" (scheduled at 2023-08-08 06:33:55.371682+00:00) notificationworker stdout | 2023-08-08 06:33:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:33:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 33, 55, 372745), True, datetime.datetime(2023, 8, 8, 6, 33, 55, 372745), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:33:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:33:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:33:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:33:55,461 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:33:55,465 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:33:55,467 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:33:55,659 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:33:55,669 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:33:55,669 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:33:55,673 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:33:56,647 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:33:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:33:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:19.687363+00:00 (in 22.796073 seconds) securityworker stdout | 2023-08-08 06:33:56,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:26 UTC)" (scheduled at 2023-08-08 06:33:56.890741+00:00) securityworker stdout | 2023-08-08 06:33:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:33:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:33:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:33:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:33:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:33:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:33:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:33:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:33:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:33:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:33:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 28, 56, 897027), 1, 2]) securityworker stdout | 2023-08-08 06:33:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 28, 56, 897027), 1, 2]) securityworker stdout | 2023-08-08 06:33:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:33:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:33:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:33:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:33:56,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:33:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:33:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:33:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:33:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:33:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:26 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:33:57,041 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:33:57,044 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:33:57,858 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:33:58,113 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:33:58,116 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:33:58,225 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:33:58,233 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:33:58,236 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:33:58,621 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:33:58,818 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:33:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:33:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:09.417660+00:00 (in 9.999435 seconds) gcworker stdout | 2023-08-08 06:33:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:09 UTC)" (scheduled at 2023-08-08 06:33:59.417660+00:00) gcworker stdout | 2023-08-08 06:33:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266839418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:33:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:33:59,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:33:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:34:00,174 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:34:00,338 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:34:01,331 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:34:02,179 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:34:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:05.371682+00:00 (in 3.192380 seconds) notificationworker stdout | 2023-08-08 06:34:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:02 UTC)" (scheduled at 2023-08-08 06:34:02.178530+00:00) notificationworker stdout | 2023-08-08 06:34:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:34:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:02 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:34:02,223 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} chunkcleanupworker stdout | 2023-08-08 06:34:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:34:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:12.293757+00:00 (in 9.999510 seconds) chunkcleanupworker stdout | 2023-08-08 06:34:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:12 UTC)" (scheduled at 2023-08-08 06:34:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:34:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:34:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 2, 294725), True, datetime.datetime(2023, 8, 8, 6, 34, 2, 294725), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:34:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:34:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:34:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:12 UTC)" executed successfully servicekey stdout | 2023-08-08 06:34:02,355 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:34:02,761 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:34:03,272 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:34:03,560 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:34:03,812 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:34:03,817 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:34:03,820 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:34:03,824 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:34:03,829 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:34:03,834 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:34:03,841 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:34:03,845 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:34:03,850 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:34:04,015 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} repositorygcworker stdout | 2023-08-08 06:34:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:34:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:09.793658+00:00 (in 5.605757 seconds) repositorygcworker stdout | 2023-08-08 06:34:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:04 UTC)" (scheduled at 2023-08-08 06:34:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:34:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:34:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:34:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:34:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:15.371682+00:00 (in 9.999433 seconds) notificationworker stdout | 2023-08-08 06:34:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:15 UTC)" (scheduled at 2023-08-08 06:34:05.371682+00:00) notificationworker stdout | 2023-08-08 06:34:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:34:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 5, 372758), True, datetime.datetime(2023, 8, 8, 6, 34, 5, 372758), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:34:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:34:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:34:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:15 UTC)" executed successfully builder stdout | 2023-08-08 06:34:05,683 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:34:05,693 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:34:05,693 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:34:05,699 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:34:07,179 [197] [DEBUG] [app] Starting request: urn:request:a263e522-4d5b-4999-8710-3e97826b71c3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:34:07,180 [199] [DEBUG] [app] Starting request: urn:request:66bc729f-3aef-4c11-a16e-572009885b74 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:34:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:07,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:07,187 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:34:07,187 [210] [DEBUG] [app] Starting request: urn:request:ae7cb42d-bdb3-40e6-a40d-84dce4ee32a4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:34:07,188 [203] [DEBUG] [app] Starting request: urn:request:2ce4f090-20aa-4588-a82b-dbcd5f4b4409 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:34:07,188 [210] [DEBUG] [app] Ending request: urn:request:ae7cb42d-bdb3-40e6-a40d-84dce4ee32a4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ae7cb42d-bdb3-40e6-a40d-84dce4ee32a4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:34:07,188 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:34:07,188 [203] [DEBUG] [app] Ending request: urn:request:2ce4f090-20aa-4588-a82b-dbcd5f4b4409 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2ce4f090-20aa-4588-a82b-dbcd5f4b4409', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:34:07,189 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:34:07,189 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:34:07,189 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:07,191 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:07,191 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:07,193 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:07,194 [197] [DEBUG] [app] Starting request: urn:request:dcb61273-ec76-4d3c-81f3-4d53c3695b6b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:34:07,195 [197] [DEBUG] [app] Ending request: urn:request:dcb61273-ec76-4d3c-81f3-4d53c3695b6b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:dcb61273-ec76-4d3c-81f3-4d53c3695b6b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:34:07,195 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:34:07,195 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:07,196 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:34:07,196 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( buildlogsarchiver stdout | 2023-08-08 06:34:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:34:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:37.197412+00:00 (in 29.999530 seconds) gunicorn-web stdout | 2023-08-08 06:34:07,197 [200] [DEBUG] [app] Starting request: urn:request:969d143d-cab0-4331-be8b-4c097e783836 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} buildlogsarchiver stdout | 2023-08-08 06:34:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:37 UTC)" (scheduled at 2023-08-08 06:34:07.197412+00:00) gunicorn-web stdout | 2023-08-08 06:34:07,198 [200] [DEBUG] [app] Ending request: urn:request:969d143d-cab0-4331-be8b-4c097e783836 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:969d143d-cab0-4331-be8b-4c097e783836', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:07,198 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:34:07,198 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 buildlogsarchiver stdout | 2023-08-08 06:34:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 34, 7, 198374), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:34:07,199 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:34:07,200 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:34:07,204 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:34:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:34:07,206 [199] [DEBUG] [app] Ending request: urn:request:66bc729f-3aef-4c11-a16e-572009885b74 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:66bc729f-3aef-4c11-a16e-572009885b74', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:34:07,206 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:34:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) gunicorn-web stdout | 2023-08-08 06:34:07,206 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:34:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:34:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:34:07,207 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:34:07,209 [197] [DEBUG] [app] Ending request: urn:request:a263e522-4d5b-4999-8710-3e97826b71c3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a263e522-4d5b-4999-8710-3e97826b71c3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:34:07,209 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:34:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.032 47 0.032) gunicorn-web stdout | 2023-08-08 06:34:07,210 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:34:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:34:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:34:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:34:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:34:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:19.417660+00:00 (in 9.999450 seconds) gcworker stdout | 2023-08-08 06:34:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:19 UTC)" (scheduled at 2023-08-08 06:34:09.417660+00:00) gcworker stdout | 2023-08-08 06:34:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266849418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:34:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:34:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:34:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:34:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:34:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:19.187230+00:00 (in 9.393063 seconds) repositorygcworker stdout | 2023-08-08 06:34:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:09 UTC)" (scheduled at 2023-08-08 06:34:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:34:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:34:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 9, 794520), True, datetime.datetime(2023, 8, 8, 6, 34, 9, 794520), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:34:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:34:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:34:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:34:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:34:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:22.293757+00:00 (in 9.999491 seconds) chunkcleanupworker stdout | 2023-08-08 06:34:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:22 UTC)" (scheduled at 2023-08-08 06:34:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:34:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:34:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 12, 294723), True, datetime.datetime(2023, 8, 8, 6, 34, 12, 294723), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:34:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:34:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:34:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:34:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:34:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:54.284093+00:00 (in 41.186187 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:34:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:13 UTC)" (scheduled at 2023-08-08 06:34:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:34:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:34:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 13, 98416), True, datetime.datetime(2023, 8, 8, 6, 34, 13, 98416), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:34:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:34:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:34:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:34:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:34:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:25.371682+00:00 (in 9.999457 seconds) notificationworker stdout | 2023-08-08 06:34:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:25 UTC)" (scheduled at 2023-08-08 06:34:15.371682+00:00) notificationworker stdout | 2023-08-08 06:34:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:34:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 15, 372857), True, datetime.datetime(2023, 8, 8, 6, 34, 15, 372857), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:34:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:34:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:34:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:25 UTC)" executed successfully builder stdout | 2023-08-08 06:34:15,709 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:34:15,719 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:34:15,719 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:34:15,723 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:34:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:34:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:04.187429+00:00 (in 44.999627 seconds) repositorygcworker stdout | 2023-08-08 06:34:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:39:19 UTC)" (scheduled at 2023-08-08 06:34:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:34:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 34, 19, 188153), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:34:19,196 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 34, 19, 188153), True, datetime.datetime(2023, 8, 8, 6, 34, 19, 188153), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:34:19,198 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 34, 19, 188153), True, datetime.datetime(2023, 8, 8, 6, 34, 19, 188153), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 34, 19, 188153), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:34:19,199 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:34:19,199 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:39:19 UTC)" executed successfully gcworker stdout | 2023-08-08 06:34:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:34:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:29.417660+00:00 (in 9.999485 seconds) gcworker stdout | 2023-08-08 06:34:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:29 UTC)" (scheduled at 2023-08-08 06:34:19.417660+00:00) gcworker stdout | 2023-08-08 06:34:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266859418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:34:19,440 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:34:19,440 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:34:19,440 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:29 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:34:19,493 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} securityworker stdout | 2023-08-08 06:34:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:34:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:26.890741+00:00 (in 7.202816 seconds) securityworker stdout | 2023-08-08 06:34:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:49 UTC)" (scheduled at 2023-08-08 06:34:19.687363+00:00) securityworker stdout | 2023-08-08 06:34:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:34:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:34:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:34:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:34:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:34:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:34:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 29, 19, 692341), 1, 2]) securityworker stdout | 2023-08-08 06:34:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 29, 19, 692341), 1, 2]) securityworker stdout | 2023-08-08 06:34:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:34:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:34:20,533 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:34:21,692 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:34:22,146 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:34:22,179 [197] [DEBUG] [app] Starting request: urn:request:5e73b386-6284-4726-8246-7719edea663d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:34:22,180 [200] [DEBUG] [app] Starting request: urn:request:7ea23385-5543-49a9-8442-a9929db87cea (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:34:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:22,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:34:22,185 [208] [DEBUG] [app] Starting request: urn:request:4bd57407-d362-4541-ad43-a503dd8b9bf0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:34:22,186 [203] [DEBUG] [app] Starting request: urn:request:6142abff-fa63-439f-9964-13d3cdd7764d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:34:22,186 [208] [DEBUG] [app] Ending request: urn:request:4bd57407-d362-4541-ad43-a503dd8b9bf0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4bd57407-d362-4541-ad43-a503dd8b9bf0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:34:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:34:22,186 [203] [DEBUG] [app] Ending request: urn:request:6142abff-fa63-439f-9964-13d3cdd7764d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6142abff-fa63-439f-9964-13d3cdd7764d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:34:22,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:34:22,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:22,192 [197] [DEBUG] [app] Starting request: urn:request:d6065d74-7631-4b41-b72b-3f2db5f6c292 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:34:22,192 [197] [DEBUG] [app] Ending request: urn:request:d6065d74-7631-4b41-b72b-3f2db5f6c292 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d6065d74-7631-4b41-b72b-3f2db5f6c292', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:22,192 [199] [DEBUG] [app] Starting request: urn:request:1b6e4c5c-f263-421c-a223-b226c2186c4d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:34:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:34:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:22,193 [199] [DEBUG] [app] Ending request: urn:request:1b6e4c5c-f263-421c-a223-b226c2186c4d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1b6e4c5c-f263-421c-a223-b226c2186c4d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:34:22,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:34:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:34:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:34:22,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:34:22,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:34:22,203 [197] [DEBUG] [app] Ending request: urn:request:5e73b386-6284-4726-8246-7719edea663d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5e73b386-6284-4726-8246-7719edea663d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:34:22,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:34:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:34:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:34:22,204 [200] [DEBUG] [app] Ending request: urn:request:7ea23385-5543-49a9-8442-a9929db87cea (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7ea23385-5543-49a9-8442-a9929db87cea', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:34:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:34:22,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:34:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:34:22,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:34:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:34:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:34:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:28.874906+00:00 (in 6.580655 seconds) chunkcleanupworker stdout | 2023-08-08 06:34:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:32 UTC)" (scheduled at 2023-08-08 06:34:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:34:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:34:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 22, 294707), True, datetime.datetime(2023, 8, 8, 6, 34, 22, 294707), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:34:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:34:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:34:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:34:24,815 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:34:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:34:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:35.371682+00:00 (in 9.999441 seconds) notificationworker stdout | 2023-08-08 06:34:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:35 UTC)" (scheduled at 2023-08-08 06:34:25.371682+00:00) notificationworker stdout | 2023-08-08 06:34:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:34:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 25, 372691), True, datetime.datetime(2023, 8, 8, 6, 34, 25, 372691), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:34:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:34:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:34:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:35 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:34:25,471 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:34:25,474 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:34:25,477 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} exportactionlogsworker stdout | 2023-08-08 06:34:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:34:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:47.398551+00:00 (in 21.883697 seconds) exportactionlogsworker stdout | 2023-08-08 06:34:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:25 UTC)" (scheduled at 2023-08-08 06:34:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:34:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:34:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:25 UTC)" executed successfully builder stdout | 2023-08-08 06:34:25,733 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:34:25,742 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:34:25,742 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:34:25,748 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:34:26,685 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:34:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:34:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:49.687363+00:00 (in 22.796085 seconds) securityworker stdout | 2023-08-08 06:34:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:56 UTC)" (scheduled at 2023-08-08 06:34:26.890741+00:00) securityworker stdout | 2023-08-08 06:34:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:34:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:34:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:34:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:34:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:34:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:34:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:34:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:34:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:34:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:34:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 29, 26, 896679), 1, 2]) securityworker stdout | 2023-08-08 06:34:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 29, 26, 896679), 1, 2]) securityworker stdout | 2023-08-08 06:34:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:34:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:34:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:34:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:34:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:34:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:34:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:34:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:34:56 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:34:27,071 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:34:27,074 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:34:27,867 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:34:28,127 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:34:28,153 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:34:28,240 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:34:28,247 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:34:28,253 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:34:28,650 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:34:28,849 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} chunkcleanupworker stdout | 2023-08-08 06:34:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:34:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:32.293757+00:00 (in 3.418311 seconds) chunkcleanupworker stdout | 2023-08-08 06:34:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:28 UTC)" (scheduled at 2023-08-08 06:34:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:34:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:34:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:28 UTC)" executed successfully gcworker stdout | 2023-08-08 06:34:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:34:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:39.417660+00:00 (in 9.999436 seconds) gcworker stdout | 2023-08-08 06:34:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:39 UTC)" (scheduled at 2023-08-08 06:34:29.417660+00:00) gcworker stdout | 2023-08-08 06:34:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266869418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:34:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:34:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:34:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:34:30,207 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:34:30,357 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:34:31,368 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} exportactionlogsworker stdout | 2023-08-08 06:34:32,261 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} chunkcleanupworker stdout | 2023-08-08 06:34:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:34:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:42.293757+00:00 (in 9.999472 seconds) chunkcleanupworker stdout | 2023-08-08 06:34:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:42 UTC)" (scheduled at 2023-08-08 06:34:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:34:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:34:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 32, 294728), True, datetime.datetime(2023, 8, 8, 6, 34, 32, 294728), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:34:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:34:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:34:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:42 UTC)" executed successfully servicekey stdout | 2023-08-08 06:34:32,390 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:34:32,795 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:34:33,295 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:34:33,597 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:34:33,822 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:34:33,827 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:34:33,830 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:34:33,834 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:34:33,838 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:34:33,842 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:34:33,852 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:34:33,855 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:34:33,859 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:34:34,046 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:34:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:34:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:45.371682+00:00 (in 9.999491 seconds) notificationworker stdout | 2023-08-08 06:34:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:45 UTC)" (scheduled at 2023-08-08 06:34:35.371682+00:00) notificationworker stdout | 2023-08-08 06:34:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:34:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 35, 372746), True, datetime.datetime(2023, 8, 8, 6, 34, 35, 372746), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:34:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:34:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:34:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:45 UTC)" executed successfully builder stdout | 2023-08-08 06:34:35,758 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:34:35,767 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:34:35,767 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:34:35,772 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:34:36,581 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:34:36,581 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:39.579576+00:00 (in 2.998403 seconds) namespacegcworker stdout | 2023-08-08 06:34:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:36 UTC)" (scheduled at 2023-08-08 06:34:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:34:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:34:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:34:37,179 [197] [DEBUG] [app] Starting request: urn:request:126561ad-e880-4704-8d11-7e2aa7b3d816 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:34:37,179 [200] [DEBUG] [app] Starting request: urn:request:4fd37c07-3e89-4053-a5be-208b8ff2604b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:34:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:34:37,185 [207] [DEBUG] [app] Starting request: urn:request:282a88e8-8d4c-4327-acb3-79068e631352 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:34:37,185 [209] [DEBUG] [app] Starting request: urn:request:021c4903-534f-4745-b374-1d6f22639afe (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:34:37,186 [207] [DEBUG] [app] Ending request: urn:request:282a88e8-8d4c-4327-acb3-79068e631352 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:282a88e8-8d4c-4327-acb3-79068e631352', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:34:37,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:34:37,186 [209] [DEBUG] [app] Ending request: urn:request:021c4903-534f-4745-b374-1d6f22639afe (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:021c4903-534f-4745-b374-1d6f22639afe', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:34:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:34:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:37,191 [199] [DEBUG] [app] Starting request: urn:request:af1dff86-b58d-40a4-8364-0bbe7ed05971 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:34:37,191 [200] [DEBUG] [app] Starting request: urn:request:1b7b69b7-f2d3-4e58-b15a-0796fe4576ab (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:34:37,192 [199] [DEBUG] [app] Ending request: urn:request:af1dff86-b58d-40a4-8364-0bbe7ed05971 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:af1dff86-b58d-40a4-8364-0bbe7ed05971', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:37,192 [200] [DEBUG] [app] Ending request: urn:request:1b7b69b7-f2d3-4e58-b15a-0796fe4576ab (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1b7b69b7-f2d3-4e58-b15a-0796fe4576ab', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:34:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:34:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:34:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:34:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:34:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:07.197412+00:00 (in 29.999529 seconds) buildlogsarchiver stdout | 2023-08-08 06:34:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:07 UTC)" (scheduled at 2023-08-08 06:34:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:34:37,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:34:37,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:34:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 34, 37, 198358), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:34:37,204 [200] [DEBUG] [app] Ending request: urn:request:4fd37c07-3e89-4053-a5be-208b8ff2604b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4fd37c07-3e89-4053-a5be-208b8ff2604b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:34:37,204 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:34:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:34:37,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:34:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:34:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:34:37,205 [197] [DEBUG] [app] Ending request: urn:request:126561ad-e880-4704-8d11-7e2aa7b3d816 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:126561ad-e880-4704-8d11-7e2aa7b3d816', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:34:37,205 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:34:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.028) gunicorn-web stdout | 2023-08-08 06:34:37,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:34:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:34:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:34:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:34:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:34:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:34:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:34:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:49.417660+00:00 (in 9.999455 seconds) gcworker stdout | 2023-08-08 06:34:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:49 UTC)" (scheduled at 2023-08-08 06:34:39.417660+00:00) gcworker stdout | 2023-08-08 06:34:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266879418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:34:39,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:34:39,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:34:39,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:34:39,579 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:34:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:54.393627+00:00 (in 14.813512 seconds) namespacegcworker stdout | 2023-08-08 06:34:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:39:39 UTC)" (scheduled at 2023-08-08 06:34:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:34:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 34, 39, 580724), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:34:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 34, 39, 580724), True, datetime.datetime(2023, 8, 8, 6, 34, 39, 580724), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:34:39,592 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 34, 39, 580724), True, datetime.datetime(2023, 8, 8, 6, 34, 39, 580724), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 34, 39, 580724), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:34:39,593 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:34:39,593 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:39:39 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:34:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:34:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:46.874674+00:00 (in 4.580420 seconds) chunkcleanupworker stdout | 2023-08-08 06:34:42,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:52 UTC)" (scheduled at 2023-08-08 06:34:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:34:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:34:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 42, 295517), True, datetime.datetime(2023, 8, 8, 6, 34, 42, 295517), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:34:42,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:34:42,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:34:42,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:34:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:34:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:55.371682+00:00 (in 9.999437 seconds) notificationworker stdout | 2023-08-08 06:34:45,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:55 UTC)" (scheduled at 2023-08-08 06:34:45.371682+00:00) notificationworker stdout | 2023-08-08 06:34:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:34:45,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 45, 373395), True, datetime.datetime(2023, 8, 8, 6, 34, 45, 373395), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:34:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:34:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:34:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:55 UTC)" executed successfully builder stdout | 2023-08-08 06:34:45,782 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:34:45,791 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:34:45,791 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:34:45,795 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:34:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:34:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:52.293757+00:00 (in 5.418604 seconds) chunkcleanupworker stdout | 2023-08-08 06:34:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:39:46 UTC)" (scheduled at 2023-08-08 06:34:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:34:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 34, 46, 875596), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:34:46,882 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 34, 46, 875596), True, datetime.datetime(2023, 8, 8, 6, 34, 46, 875596), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:34:46,884 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 34, 46, 875596), True, datetime.datetime(2023, 8, 8, 6, 34, 46, 875596), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 34, 46, 875596), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:34:46,886 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:34:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:39:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:34:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:34:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:25.514341+00:00 (in 38.115267 seconds) exportactionlogsworker stdout | 2023-08-08 06:34:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:47 UTC)" (scheduled at 2023-08-08 06:34:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:34:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:34:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 47, 399670), True, datetime.datetime(2023, 8, 8, 6, 34, 47, 399670), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:34:47,408 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:34:47,408 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:34:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:34:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:34:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:59.417660+00:00 (in 9.999509 seconds) gcworker stdout | 2023-08-08 06:34:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:59 UTC)" (scheduled at 2023-08-08 06:34:49.417660+00:00) gcworker stdout | 2023-08-08 06:34:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266889418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:34:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:34:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:34:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:34:59 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:34:49,530 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} securityworker stdout | 2023-08-08 06:34:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:34:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:34:56.890741+00:00 (in 7.202853 seconds) securityworker stdout | 2023-08-08 06:34:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:19 UTC)" (scheduled at 2023-08-08 06:34:49.687363+00:00) securityworker stdout | 2023-08-08 06:34:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:34:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:34:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:34:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:34:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:34:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:34:50,570 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:34:51,721 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:34:52,177 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:34:52,179 [197] [DEBUG] [app] Starting request: urn:request:296013fd-5447-45e9-b334-fb370fb14b23 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:34:52,180 [200] [DEBUG] [app] Starting request: urn:request:cb621f94-f25a-49c6-87de-ec199168d376 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:34:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:52,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:34:52,185 [203] [DEBUG] [app] Starting request: urn:request:159c0325-8957-4cef-8591-6a2ccc745d4e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:34:52,186 [203] [DEBUG] [app] Ending request: urn:request:159c0325-8957-4cef-8591-6a2ccc745d4e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:159c0325-8957-4cef-8591-6a2ccc745d4e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:34:52,186 [209] [DEBUG] [app] Starting request: urn:request:edfd1ca9-cde4-4694-905a-8679e61c8e9a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:34:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:34:52,186 [209] [DEBUG] [app] Ending request: urn:request:edfd1ca9-cde4-4694-905a-8679e61c8e9a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:edfd1ca9-cde4-4694-905a-8679e61c8e9a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:34:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:34:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:34:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:52,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:34:52,191 [200] [DEBUG] [app] Starting request: urn:request:e4ca25c1-9a0d-48e5-86ff-7cda46ccaffc (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:34:52,192 [200] [DEBUG] [app] Ending request: urn:request:e4ca25c1-9a0d-48e5-86ff-7cda46ccaffc (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e4ca25c1-9a0d-48e5-86ff-7cda46ccaffc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:34:52,192 [198] [DEBUG] [app] Starting request: urn:request:6df12b71-498c-4038-a1f0-b91b300ad99d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:34:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:52,193 [198] [DEBUG] [app] Ending request: urn:request:6df12b71-498c-4038-a1f0-b91b300ad99d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6df12b71-498c-4038-a1f0-b91b300ad99d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:34:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:34:52,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:34:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:34:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:34:52,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:34:52,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:34:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:34:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:34:52,203 [200] [DEBUG] [app] Ending request: urn:request:cb621f94-f25a-49c6-87de-ec199168d376 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cb621f94-f25a-49c6-87de-ec199168d376', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:34:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:34:52,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:34:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:34:52,204 [197] [DEBUG] [app] Ending request: urn:request:296013fd-5447-45e9-b334-fb370fb14b23 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:296013fd-5447-45e9-b334-fb370fb14b23', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:34:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:34:52,204 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:34:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) gunicorn-web stdout | 2023-08-08 06:34:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:34:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:34:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:34:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:02.293757+00:00 (in 9.999493 seconds) chunkcleanupworker stdout | 2023-08-08 06:34:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:02 UTC)" (scheduled at 2023-08-08 06:34:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:34:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:34:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 52, 294816), True, datetime.datetime(2023, 8, 8, 6, 34, 52, 294816), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:34:52,345 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:34:52,346 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:34:52,346 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:34:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:34:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:13.097375+00:00 (in 18.812774 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:34:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:54 UTC)" (scheduled at 2023-08-08 06:34:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:34:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:34:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:34:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:34:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:36.579841+00:00 (in 42.185710 seconds) namespacegcworker stdout | 2023-08-08 06:34:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:54 UTC)" (scheduled at 2023-08-08 06:34:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:34:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:34:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 54, 394603), True, datetime.datetime(2023, 8, 8, 6, 34, 54, 394603), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:34:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:34:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:34:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:35:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:34:54,849 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:34:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:34:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:02.178530+00:00 (in 6.806270 seconds) notificationworker stdout | 2023-08-08 06:34:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:05 UTC)" (scheduled at 2023-08-08 06:34:55.371682+00:00) notificationworker stdout | 2023-08-08 06:34:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:34:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 34, 55, 372719), True, datetime.datetime(2023, 8, 8, 6, 34, 55, 372719), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:34:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:34:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:34:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:34:55,481 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:34:55,485 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:34:55,489 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:34:55,798 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:34:55,807 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:34:55,807 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:34:55,812 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:34:56,722 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:34:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:34:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:19.687363+00:00 (in 22.796074 seconds) securityworker stdout | 2023-08-08 06:34:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:26 UTC)" (scheduled at 2023-08-08 06:34:56.890741+00:00) securityworker stdout | 2023-08-08 06:34:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:34:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:34:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:34:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:34:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:34:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:34:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:34:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:34:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:34:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:34:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 29, 56, 896439), 1, 2]) securityworker stdout | 2023-08-08 06:34:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 29, 56, 896439), 1, 2]) securityworker stdout | 2023-08-08 06:34:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:34:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:34:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:34:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:34:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:34:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:34:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:34:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:34:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:34:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:26 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:34:57,096 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:34:57,099 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:34:57,876 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:34:58,141 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:34:58,180 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:34:58,253 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:34:58,264 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:34:58,267 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:34:58,666 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:34:58,863 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:34:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:34:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:09.417660+00:00 (in 9.999446 seconds) gcworker stdout | 2023-08-08 06:34:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:09 UTC)" (scheduled at 2023-08-08 06:34:59.417660+00:00) gcworker stdout | 2023-08-08 06:34:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266899418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:34:59,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:34:59,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:34:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:35:00,242 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:35:00,394 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:35:01,389 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:35:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:35:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:05.371682+00:00 (in 3.192577 seconds) notificationworker stdout | 2023-08-08 06:35:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:02 UTC)" (scheduled at 2023-08-08 06:35:02.178530+00:00) notificationworker stdout | 2023-08-08 06:35:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:35:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:35:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:35:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:12.293757+00:00 (in 9.999450 seconds) chunkcleanupworker stdout | 2023-08-08 06:35:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:12 UTC)" (scheduled at 2023-08-08 06:35:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:35:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:35:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 2, 294928), True, datetime.datetime(2023, 8, 8, 6, 35, 2, 294928), 0, 'chunk_cleanup/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:35:02,299 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} chunkcleanupworker stdout | 2023-08-08 06:35:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:35:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:35:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:12 UTC)" executed successfully servicekey stdout | 2023-08-08 06:35:02,428 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:35:02,813 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:35:03,303 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:35:03,629 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:35:03,832 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:35:03,836 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:35:03,840 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:35:03,843 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:35:03,847 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:35:03,850 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:35:03,863 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:35:03,866 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:35:03,869 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:35:04,083 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} repositorygcworker stdout | 2023-08-08 06:35:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:35:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:09.793658+00:00 (in 5.605724 seconds) repositorygcworker stdout | 2023-08-08 06:35:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:04 UTC)" (scheduled at 2023-08-08 06:35:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:35:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:35:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:35:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:35:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:15.371682+00:00 (in 9.999527 seconds) notificationworker stdout | 2023-08-08 06:35:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:15 UTC)" (scheduled at 2023-08-08 06:35:05.371682+00:00) notificationworker stdout | 2023-08-08 06:35:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:35:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 5, 372495), True, datetime.datetime(2023, 8, 8, 6, 35, 5, 372495), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:35:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:35:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:35:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:15 UTC)" executed successfully builder stdout | 2023-08-08 06:35:05,819 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:35:05,830 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:35:05,830 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:35:05,835 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:35:07,179 [197] [DEBUG] [app] Starting request: urn:request:0609bbd7-6bba-4b31-8e83-c6d08fedaa1b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:35:07,179 [199] [DEBUG] [app] Starting request: urn:request:41f37e47-5d98-45c5-9e4b-71a6ed269e8f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:35:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:07,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:35:07,184 [203] [DEBUG] [app] Starting request: urn:request:15917e2b-d30b-44bd-a9c7-ec1128622275 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:35:07,185 [203] [DEBUG] [app] Ending request: urn:request:15917e2b-d30b-44bd-a9c7-ec1128622275 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:15917e2b-d30b-44bd-a9c7-ec1128622275', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:35:07,185 [209] [DEBUG] [app] Starting request: urn:request:7e7d0d68-cc46-4b3d-a815-9709b40b6187 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:35:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:35:07,185 [209] [DEBUG] [app] Ending request: urn:request:7e7d0d68-cc46-4b3d-a815-9709b40b6187 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7e7d0d68-cc46-4b3d-a815-9709b40b6187', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:07,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:35:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:35:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:07,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:07,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:07,190 [198] [DEBUG] [app] Starting request: urn:request:3558ed09-f581-46c3-8237-de17fba84d12 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:07,191 [198] [DEBUG] [app] Ending request: urn:request:3558ed09-f581-46c3-8237-de17fba84d12 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3558ed09-f581-46c3-8237-de17fba84d12', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:07,191 [200] [DEBUG] [app] Starting request: urn:request:aa904fa6-23dc-44fb-b937-94935856c36b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:35:07,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:35:07,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:07,191 [200] [DEBUG] [app] Ending request: urn:request:aa904fa6-23dc-44fb-b937-94935856c36b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:aa904fa6-23dc-44fb-b937-94935856c36b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:07,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:35:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:35:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:35:07,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:35:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:35:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:35:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:37.197412+00:00 (in 29.999478 seconds) buildlogsarchiver stdout | 2023-08-08 06:35:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:37 UTC)" (scheduled at 2023-08-08 06:35:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:35:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 35, 7, 198747), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:07,203 [197] [DEBUG] [app] Ending request: urn:request:0609bbd7-6bba-4b31-8e83-c6d08fedaa1b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0609bbd7-6bba-4b31-8e83-c6d08fedaa1b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:35:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:35:07,203 [199] [DEBUG] [app] Ending request: urn:request:41f37e47-5d98-45c5-9e4b-71a6ed269e8f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:41f37e47-5d98-45c5-9e4b-71a6ed269e8f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:35:07,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:35:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:35:07,204 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:35:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:35:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:35:07,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:35:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:35:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:35:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:35:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:35:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:35:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:35:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:35:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:19.417660+00:00 (in 9.999516 seconds) gcworker stdout | 2023-08-08 06:35:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:19 UTC)" (scheduled at 2023-08-08 06:35:09.417660+00:00) gcworker stdout | 2023-08-08 06:35:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266909418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:35:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:35:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:35:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:35:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:35:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:04.187429+00:00 (in 54.393295 seconds) repositorygcworker stdout | 2023-08-08 06:35:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:09 UTC)" (scheduled at 2023-08-08 06:35:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:35:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:35:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 9, 794521), True, datetime.datetime(2023, 8, 8, 6, 35, 9, 794521), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:35:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:35:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:35:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:35:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:35:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:22.293757+00:00 (in 9.999460 seconds) chunkcleanupworker stdout | 2023-08-08 06:35:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:22 UTC)" (scheduled at 2023-08-08 06:35:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:35:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:35:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 12, 294902), True, datetime.datetime(2023, 8, 8, 6, 35, 12, 294902), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:35:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:35:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:35:12,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:35:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:35:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:54.284093+00:00 (in 41.186202 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:35:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:13 UTC)" (scheduled at 2023-08-08 06:35:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:35:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:35:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 13, 98362), True, datetime.datetime(2023, 8, 8, 6, 35, 13, 98362), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:35:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:35:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:35:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:35:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:35:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:25.371682+00:00 (in 9.999548 seconds) notificationworker stdout | 2023-08-08 06:35:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:25 UTC)" (scheduled at 2023-08-08 06:35:15.371682+00:00) notificationworker stdout | 2023-08-08 06:35:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:35:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 15, 372472), True, datetime.datetime(2023, 8, 8, 6, 35, 15, 372472), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:35:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:35:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:35:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:25 UTC)" executed successfully builder stdout | 2023-08-08 06:35:15,846 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:35:15,854 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:35:15,855 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:35:15,859 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:35:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:35:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:29.417660+00:00 (in 9.999487 seconds) gcworker stdout | 2023-08-08 06:35:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:29 UTC)" (scheduled at 2023-08-08 06:35:19.417660+00:00) gcworker stdout | 2023-08-08 06:35:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266919418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:35:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:35:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:35:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:29 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:35:19,568 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} securityworker stdout | 2023-08-08 06:35:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:35:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:26.890741+00:00 (in 7.202885 seconds) securityworker stdout | 2023-08-08 06:35:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:49 UTC)" (scheduled at 2023-08-08 06:35:19.687363+00:00) securityworker stdout | 2023-08-08 06:35:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:35:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:35:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:35:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:35:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:35:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:35:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 30, 19, 691940), 1, 2]) securityworker stdout | 2023-08-08 06:35:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 30, 19, 691940), 1, 2]) securityworker stdout | 2023-08-08 06:35:19,706 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:35:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:35:20,592 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:35:21,748 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:35:22,179 [199] [DEBUG] [app] Starting request: urn:request:102c0696-467e-4ba5-bd62-f66450aa76a5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:35:22,179 [200] [DEBUG] [app] Starting request: urn:request:1b693701-730d-4c5d-89b1-26878b954e95 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:35:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:35:22,184 [203] [DEBUG] [app] Starting request: urn:request:ed32a764-a877-4df8-9a3a-4425bd88a53e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:35:22,185 [210] [DEBUG] [app] Starting request: urn:request:06bf7d62-c718-4c62-8793-e5424b1f149b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:35:22,185 [203] [DEBUG] [app] Ending request: urn:request:ed32a764-a877-4df8-9a3a-4425bd88a53e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ed32a764-a877-4df8-9a3a-4425bd88a53e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:35:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:35:22,185 [210] [DEBUG] [app] Ending request: urn:request:06bf7d62-c718-4c62-8793-e5424b1f149b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:06bf7d62-c718-4c62-8793-e5424b1f149b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:35:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:35:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:35:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:22,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:22,190 [198] [DEBUG] [app] Starting request: urn:request:d5ec607d-1c2f-48ec-8379-68d7e77310d9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:22,191 [200] [DEBUG] [app] Starting request: urn:request:c5f96083-0666-442b-bf75-d8c9e34ca647 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:22,191 [198] [DEBUG] [app] Ending request: urn:request:d5ec607d-1c2f-48ec-8379-68d7e77310d9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d5ec607d-1c2f-48ec-8379-68d7e77310d9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:22,191 [200] [DEBUG] [app] Ending request: urn:request:c5f96083-0666-442b-bf75-d8c9e34ca647 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c5f96083-0666-442b-bf75-d8c9e34ca647', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:22,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:35:22,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:22,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:35:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:35:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:35:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:35:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:35:22,203 [199] [DEBUG] [app] Ending request: urn:request:102c0696-467e-4ba5-bd62-f66450aa76a5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:102c0696-467e-4ba5-bd62-f66450aa76a5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:35:22,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:35:22,203 [200] [DEBUG] [app] Ending request: urn:request:1b693701-730d-4c5d-89b1-26878b954e95 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1b693701-730d-4c5d-89b1-26878b954e95', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:35:22,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:35:22,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:35:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:35:22,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:35:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:35:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:35:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) storagereplication stdout | 2023-08-08 06:35:22,214 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} chunkcleanupworker stdout | 2023-08-08 06:35:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:35:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:28.874906+00:00 (in 6.580557 seconds) chunkcleanupworker stdout | 2023-08-08 06:35:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:32 UTC)" (scheduled at 2023-08-08 06:35:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:35:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:35:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 22, 294705), True, datetime.datetime(2023, 8, 8, 6, 35, 22, 294705), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:35:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:35:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:35:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:35:24,886 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:35:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:35:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:35.371682+00:00 (in 9.999505 seconds) notificationworker stdout | 2023-08-08 06:35:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:35 UTC)" (scheduled at 2023-08-08 06:35:25.371682+00:00) notificationworker stdout | 2023-08-08 06:35:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:35:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 25, 372511), True, datetime.datetime(2023, 8, 8, 6, 35, 25, 372511), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:35:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:35:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:35:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:35 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:35:25,490 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:35:25,493 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:35:25,497 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} exportactionlogsworker stdout | 2023-08-08 06:35:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:35:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:47.398551+00:00 (in 21.883558 seconds) exportactionlogsworker stdout | 2023-08-08 06:35:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:25 UTC)" (scheduled at 2023-08-08 06:35:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:35:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:35:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:25 UTC)" executed successfully builder stdout | 2023-08-08 06:35:25,861 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:35:25,870 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:35:25,870 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:35:25,875 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:35:26,759 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:35:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:35:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:49.687363+00:00 (in 22.796120 seconds) securityworker stdout | 2023-08-08 06:35:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:56 UTC)" (scheduled at 2023-08-08 06:35:26.890741+00:00) securityworker stdout | 2023-08-08 06:35:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:35:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:35:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:35:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:35:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:35:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:35:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:35:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:35:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:35:26,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:35:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 30, 26, 896261), 1, 2]) securityworker stdout | 2023-08-08 06:35:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 30, 26, 896261), 1, 2]) securityworker stdout | 2023-08-08 06:35:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:35:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:35:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:35:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:35:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:35:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:35:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:35:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:35:56 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:35:27,133 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:35:27,136 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:35:27,687 [197] [DEBUG] [app] Starting request: urn:request:e8de6880-9571-4913-b599-2a9cdb15e755 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:35:27,687 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:35:27,687 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:35:27,688 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:27,748 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:35:27,749 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:35:27,749 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:35:27,749 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/github gunicorn-web stdout | 2023-08-08 06:35:27,749 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:35:27,749 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:35:27,749 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:35:27,749 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:35:27,751 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'user1_build_test', 3, 'github', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:35:27,783 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='github', role='admin') gunicorn-web stdout | 2023-08-08 06:35:27,784 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:27,787 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:27,790 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['58f84ee1-b064-4be3-aeb1-b8299deea05c', 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:27,794 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:27,795 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:35:27,795 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['write', 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:27,797 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:27,799 [197] [DEBUG] [peewee] ('INSERT INTO "accesstoken" ("friendly_name", "token_name", "token_code", "repository_id", "created", "role_id", "temporary") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "accesstoken"."id"', ['Build Trigger: gitlab', 'I5EJCLYAO5N8O43KGBRYCXSJ2J1GHI1F', 'v0$$SdyfbGVPV3w1cGEJxdt+gUzELcbuYxSTUoo8IhhbPXP1Mns+prnCFu4zG/sljYd65GzubQP1UzhKDPmcRQ==', 4, datetime.datetime(2023, 8, 8, 6, 35, 27, 798993), 2, False]) gunicorn-web stdout | 2023-08-08 06:35:27,805 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:35:27,885 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:35:28,083 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:35:28,157 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:35:28,204 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:35:28,268 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:35:28,279 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:35:28,282 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:35:28,372 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:35:28,656 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "POST /api/v4/projects/35911436/deploy_keys HTTP/1.1" 201 673 repositoryactioncounter stdout | 2023-08-08 06:35:28,681 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} chunkcleanupworker stdout | 2023-08-08 06:35:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:35:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:32.293757+00:00 (in 3.418169 seconds) chunkcleanupworker stdout | 2023-08-08 06:35:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:28 UTC)" (scheduled at 2023-08-08 06:35:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:35:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:35:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:28 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:35:28,882 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:35:28,900 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "POST /api/v4/projects/35911436/hooks HTTP/1.1" 422 29 gunicorn-web stdout | 2023-08-08 06:35:28,901 [197] [ERROR] [buildtrigger.gitlabhandler] GitLab API error. Please contact support. gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 267, in wrapped_f gunicorn-web stdout | return f(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/mixins.py", line 211, in create gunicorn-web stdout | server_data = self.gitlab.http_post(path, post_data=data, files=files, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 680, in http_post gunicorn-web stdout | result = self.http_request( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 573, in http_request gunicorn-web stdout | raise GitlabHttpError( gunicorn-web stdout | gitlab.exceptions.GitlabHttpError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 100, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 270, in activate gunicorn-web stdout | hook = gl_project.hooks.create( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 269, in wrapped_f gunicorn-web stdout | raise error(e.error_message, e.response_code, e.response_body) gunicorn-web stdout | gitlab.exceptions.GitlabCreateError: 422: Invalid url given gunicorn-web stdout | 2023-08-08 06:35:28,902 [197] [ERROR] [app] Exception on /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate [POST] gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 267, in wrapped_f gunicorn-web stdout | return f(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/mixins.py", line 211, in create gunicorn-web stdout | server_data = self.gitlab.http_post(path, post_data=data, files=files, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 680, in http_post gunicorn-web stdout | result = self.http_request( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 573, in http_request gunicorn-web stdout | raise GitlabHttpError( gunicorn-web stdout | gitlab.exceptions.GitlabHttpError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 100, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 270, in activate gunicorn-web stdout | hook = gl_project.hooks.create( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 269, in wrapped_f gunicorn-web stdout | raise error(e.error_message, e.response_code, e.response_body) gunicorn-web stdout | gitlab.exceptions.GitlabCreateError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/app.py", line 1949, in full_dispatch_request gunicorn-web stdout | rv = self.dispatch_request() gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/app.py", line 1935, in dispatch_request gunicorn-web stdout | return self.view_functions[rule.endpoint](**req.view_args) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 256, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/auth/decorators.py", line 65, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/util/request.py", line 50, in wrapper gunicorn-web stdout | return decorator(func)(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/utils/cors.py", line 35, in wrapped_function gunicorn-web stdout | resp = make_response(f(*args, **kwargs)) gunicorn-web stdout | File "/quay-registry/endpoints/csrf.py", line 73, in wrapper gunicorn-web stdout | resp = func(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/__init__.py", line 467, in wrapper gunicorn-web stdout | resp = resource(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/views.py", line 89, in view gunicorn-web stdout | return self.dispatch_request(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/__init__.py", line 582, in dispatch_request gunicorn-web stdout | resp = meth(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 189, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 247, in wrapper gunicorn-web stdout | return func(namespace, repository, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 164, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 342, in wrapped gunicorn-web stdout | return func(self, namespace, repository, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 296, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 308, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 283, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 494, in wrapped gunicorn-web stdout | return func(self, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/trigger.py", line 323, in post gunicorn-web stdout | final_config, private_config = handler.activate(authed_url) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 108, in wrapper gunicorn-web stdout | raise ExternalServiceError(msg) gunicorn-web stdout | endpoints.exception.ExternalServiceError: 520 Unknown Error: GitLab API error. Please contact support. gunicorn-web stdout | 2023-08-08 06:35:28,903 [197] [DEBUG] [app] Ending request: urn:request:e8de6880-9571-4913-b599-2a9cdb15e755 (/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate) {'endpoint': 'api.buildtriggeractivate', 'request_id': 'urn:request:e8de6880-9571-4913-b599-2a9cdb15e755', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate', 'path': '/api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate', 'parameters': {}, 'json_body': {'config': {'build_source': 'h4051/quay_build_test', 'dockerfile_path': '/Dockerfile', 'context': '/', 'default_tag_from_ref': True, 'latest_for_default_branch': True, 'tag_templates': [], 'credentials': [{'name': 'SSH Public Key', 'value': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCiyhyEddkbHiiyX+4baphBksRzWiVaPF2uNM5LUQfHpTZWJwhOdwFlz17W1K7CE79KSiK9gl8hgnbYSL41Hcf513rhPYcIQmK/htHI9qVlvfbdLOTk5pY270GOoxxFVJ2UejRwMXOHj6ignyh387mQX2RkF39byD3Bz35ZtSPi05hYRMt7bNovWikZ/17ZTXs23uT5jifJci21URSKa0rzkjTmEn4oyJpAfRC0NvG8iazBColLEK4UbZy7z6yIl55m63iFygUjCZ70SBDvZk02Sn9xyqzf9UVgsZS/aoE+CddTV4PhHy7KVG2daGyTqZrlaDkLqhXcQiRdyuyC8C2F'}], 'key_id': 12098893}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:35:28,904 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:35:28 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate HTTP/2.0" 520 348 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.219 1202 1.220) gunicorn-web stdout | 2023-08-08 06:35:28,904 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:35:28 +0000] "POST /api/v1/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c/activate HTTP/1.0" 520 348 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gcworker stdout | 2023-08-08 06:35:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:35:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:39.417660+00:00 (in 9.999409 seconds) gcworker stdout | 2023-08-08 06:35:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:39 UTC)" (scheduled at 2023-08-08 06:35:29.417660+00:00) gcworker stdout | 2023-08-08 06:35:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266929418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:35:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:35:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:35:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:35:30,254 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:35:30,403 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:35:31,420 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:35:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:35:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:42.293757+00:00 (in 9.999443 seconds) chunkcleanupworker stdout | 2023-08-08 06:35:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:42 UTC)" (scheduled at 2023-08-08 06:35:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:35:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:35:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 32, 294696), True, datetime.datetime(2023, 8, 8, 6, 35, 32, 294696), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:35:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:35:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:35:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:35:32,322 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:35:32,438 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:35:32,850 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:35:33,340 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:35:33,660 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:35:33,841 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:35:33,846 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:35:33,849 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:35:33,851 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:35:33,855 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:35:33,858 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:35:33,874 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:35:33,879 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:35:33,882 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:35:34,105 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:35:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:35:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:45.371682+00:00 (in 9.999537 seconds) notificationworker stdout | 2023-08-08 06:35:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:45 UTC)" (scheduled at 2023-08-08 06:35:35.371682+00:00) notificationworker stdout | 2023-08-08 06:35:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:35:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 35, 372481), True, datetime.datetime(2023, 8, 8, 6, 35, 35, 372481), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:35:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:35:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:35:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:45 UTC)" executed successfully builder stdout | 2023-08-08 06:35:35,881 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:35:35,892 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:35:35,892 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:35:35,896 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:35:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:35:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:54.393627+00:00 (in 17.813275 seconds) namespacegcworker stdout | 2023-08-08 06:35:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:36 UTC)" (scheduled at 2023-08-08 06:35:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:35:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:35:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:35:37,179 [199] [DEBUG] [app] Starting request: urn:request:18e160b7-cb7d-4154-a0e1-a0a6f0f44192 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:35:37,180 [200] [DEBUG] [app] Starting request: urn:request:399b8445-87f0-40d0-958b-3f48ab9a2835 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:35:37,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:37,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:37,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:35:37,185 [208] [DEBUG] [app] Starting request: urn:request:bee27812-a207-46f3-a399-f5c48de48261 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:35:37,185 [208] [DEBUG] [app] Ending request: urn:request:bee27812-a207-46f3-a399-f5c48de48261 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:bee27812-a207-46f3-a399-f5c48de48261', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:35:37,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:35:37,186 [203] [DEBUG] [app] Starting request: urn:request:a44a0a6d-4c5b-4217-88a6-bbb6e34df2dc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:37,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:35:37,186 [203] [DEBUG] [app] Ending request: urn:request:a44a0a6d-4c5b-4217-88a6-bbb6e34df2dc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a44a0a6d-4c5b-4217-88a6-bbb6e34df2dc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:35:37,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:35:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:37,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:37,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:37,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:37,191 [197] [DEBUG] [app] Starting request: urn:request:6603f602-0a60-497a-b029-e6a776b29782 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:37,192 [199] [DEBUG] [app] Starting request: urn:request:1c61d454-1341-4fdf-a511-5227450127ed (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:37,192 [197] [DEBUG] [app] Ending request: urn:request:6603f602-0a60-497a-b029-e6a776b29782 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6603f602-0a60-497a-b029-e6a776b29782', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:37,192 [199] [DEBUG] [app] Ending request: urn:request:1c61d454-1341-4fdf-a511-5227450127ed (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1c61d454-1341-4fdf-a511-5227450127ed', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:35:37,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:35:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:37,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:37,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:35:37,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:35:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:35:37,197 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:07.197412+00:00 (in 29.999640 seconds) buildlogsarchiver stdout | 2023-08-08 06:35:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:07 UTC)" (scheduled at 2023-08-08 06:35:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:35:37,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:35:37,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:35:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 35, 37, 198605), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:35:37,204 [199] [DEBUG] [app] Ending request: urn:request:18e160b7-cb7d-4154-a0e1-a0a6f0f44192 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:18e160b7-cb7d-4154-a0e1-a0a6f0f44192', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:35:37,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:35:37,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:35:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:35:37,205 [200] [DEBUG] [app] Ending request: urn:request:399b8445-87f0-40d0-958b-3f48ab9a2835 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:399b8445-87f0-40d0-958b-3f48ab9a2835', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:35:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 06:35:37,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:35:37,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:35:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:35:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:35:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:35:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:35:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:35:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:35:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:35:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:35:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:49.417660+00:00 (in 9.999484 seconds) gcworker stdout | 2023-08-08 06:35:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:49 UTC)" (scheduled at 2023-08-08 06:35:39.417660+00:00) gcworker stdout | 2023-08-08 06:35:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266939418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:35:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:35:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:35:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:35:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:35:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:52.293757+00:00 (in 9.999462 seconds) chunkcleanupworker stdout | 2023-08-08 06:35:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:52 UTC)" (scheduled at 2023-08-08 06:35:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:35:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:35:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 42, 294675), True, datetime.datetime(2023, 8, 8, 6, 35, 42, 294675), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:35:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:35:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:35:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:35:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:35:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:55.371682+00:00 (in 9.999546 seconds) notificationworker stdout | 2023-08-08 06:35:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:55 UTC)" (scheduled at 2023-08-08 06:35:45.371682+00:00) notificationworker stdout | 2023-08-08 06:35:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:35:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 45, 372572), True, datetime.datetime(2023, 8, 8, 6, 35, 45, 372572), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:35:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:35:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:35:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:55 UTC)" executed successfully builder stdout | 2023-08-08 06:35:45,904 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:35:45,913 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:35:45,913 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:35:45,917 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:35:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:35:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:25.514341+00:00 (in 38.115260 seconds) exportactionlogsworker stdout | 2023-08-08 06:35:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:47 UTC)" (scheduled at 2023-08-08 06:35:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:35:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:35:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 47, 399697), True, datetime.datetime(2023, 8, 8, 6, 35, 47, 399697), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:35:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:35:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:35:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:35:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:35:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:59.417660+00:00 (in 9.999450 seconds) gcworker stdout | 2023-08-08 06:35:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:59 UTC)" (scheduled at 2023-08-08 06:35:49.417660+00:00) gcworker stdout | 2023-08-08 06:35:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266949418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:35:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:35:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:35:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:35:59 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:35:49,589 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} securityworker stdout | 2023-08-08 06:35:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:35:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:35:56.890741+00:00 (in 7.202873 seconds) securityworker stdout | 2023-08-08 06:35:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:19 UTC)" (scheduled at 2023-08-08 06:35:49.687363+00:00) securityworker stdout | 2023-08-08 06:35:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:35:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:35:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:35:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:35:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:35:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:35:50,608 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:35:51,771 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:35:52,179 [197] [DEBUG] [app] Starting request: urn:request:a5c4e5cc-6e8a-4ed4-8413-17ced99747d8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:35:52,179 [200] [DEBUG] [app] Starting request: urn:request:cb232e90-fbda-47f7-aef6-0b4aa6b51221 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:35:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:35:52,185 [208] [DEBUG] [app] Starting request: urn:request:e0b7b86b-49a8-4f9c-acb7-1743c23ef1b2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:35:52,186 [203] [DEBUG] [app] Starting request: urn:request:cd20f063-2089-46ba-ad77-5d7693691295 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:35:52,186 [208] [DEBUG] [app] Ending request: urn:request:e0b7b86b-49a8-4f9c-acb7-1743c23ef1b2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e0b7b86b-49a8-4f9c-acb7-1743c23ef1b2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:35:52,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:35:52,186 [203] [DEBUG] [app] Ending request: urn:request:cd20f063-2089-46ba-ad77-5d7693691295 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:cd20f063-2089-46ba-ad77-5d7693691295', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:35:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:35:52,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:35:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:35:52,191 [199] [DEBUG] [app] Starting request: urn:request:7b639e92-ab4b-4ce7-bf9b-61b4b6cbfdf4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:52,191 [197] [DEBUG] [app] Starting request: urn:request:a4b4afe2-8328-4fcf-9dfb-7980070bb57f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:35:52,192 [199] [DEBUG] [app] Ending request: urn:request:7b639e92-ab4b-4ce7-bf9b-61b4b6cbfdf4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7b639e92-ab4b-4ce7-bf9b-61b4b6cbfdf4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:52,192 [197] [DEBUG] [app] Ending request: urn:request:a4b4afe2-8328-4fcf-9dfb-7980070bb57f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a4b4afe2-8328-4fcf-9dfb-7980070bb57f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:35:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:35:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:35:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:35:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:35:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:35:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:35:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:35:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:35:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:35:52,203 [200] [DEBUG] [app] Ending request: urn:request:cb232e90-fbda-47f7-aef6-0b4aa6b51221 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cb232e90-fbda-47f7-aef6-0b4aa6b51221', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:35:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:35:52,203 [197] [DEBUG] [app] Ending request: urn:request:a5c4e5cc-6e8a-4ed4-8413-17ced99747d8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a5c4e5cc-6e8a-4ed4-8413-17ced99747d8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:35:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:35:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:35:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:35:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:35:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:35:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:35:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) storagereplication stdout | 2023-08-08 06:35:52,251 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} chunkcleanupworker stdout | 2023-08-08 06:35:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:35:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:02.293757+00:00 (in 9.999460 seconds) chunkcleanupworker stdout | 2023-08-08 06:35:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:02 UTC)" (scheduled at 2023-08-08 06:35:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:35:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:35:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 52, 294700), True, datetime.datetime(2023, 8, 8, 6, 35, 52, 294700), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:35:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:35:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:35:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:02 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:35:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:35:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:13.097375+00:00 (in 18.812703 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:35:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:54 UTC)" (scheduled at 2023-08-08 06:35:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:35:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:35:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:35:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:35:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:36.579841+00:00 (in 42.185745 seconds) namespacegcworker stdout | 2023-08-08 06:35:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:54 UTC)" (scheduled at 2023-08-08 06:35:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:35:54,395 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:35:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 54, 395052), True, datetime.datetime(2023, 8, 8, 6, 35, 54, 395052), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:35:54,404 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:35:54,404 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:35:54,404 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:36:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:35:54,923 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:35:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:35:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:02.178530+00:00 (in 6.806389 seconds) notificationworker stdout | 2023-08-08 06:35:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:05 UTC)" (scheduled at 2023-08-08 06:35:55.371682+00:00) notificationworker stdout | 2023-08-08 06:35:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:35:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 35, 55, 373156), True, datetime.datetime(2023, 8, 8, 6, 35, 55, 373156), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:35:55,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:35:55,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:35:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:35:55,499 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:35:55,502 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:35:55,505 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:35:55,928 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:35:55,937 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:35:55,937 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:35:55,941 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:35:56,797 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:35:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:35:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:19.687363+00:00 (in 22.796120 seconds) securityworker stdout | 2023-08-08 06:35:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:26 UTC)" (scheduled at 2023-08-08 06:35:56.890741+00:00) securityworker stdout | 2023-08-08 06:35:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:35:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:35:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:35:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:35:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:35:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:35:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:35:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:35:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:35:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:35:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 30, 56, 896435), 1, 2]) securityworker stdout | 2023-08-08 06:35:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 30, 56, 896435), 1, 2]) securityworker stdout | 2023-08-08 06:35:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:35:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:35:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:35:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:35:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:35:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:35:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:35:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:35:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:35:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:26 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:35:57,166 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:35:57,171 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:35:57,894 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:35:58,170 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:35:58,213 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:35:58,281 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:35:58,293 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:35:58,298 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:35:58,718 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:35:58,919 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:35:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:35:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:09.417660+00:00 (in 9.999521 seconds) gcworker stdout | 2023-08-08 06:35:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:09 UTC)" (scheduled at 2023-08-08 06:35:59.417660+00:00) gcworker stdout | 2023-08-08 06:35:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266959418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:35:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:35:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:35:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:36:00,292 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:36:00,429 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:36:01,451 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:36:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:36:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:05.371682+00:00 (in 3.192675 seconds) notificationworker stdout | 2023-08-08 06:36:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:02 UTC)" (scheduled at 2023-08-08 06:36:02.178530+00:00) notificationworker stdout | 2023-08-08 06:36:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:36:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:36:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:36:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:12.293757+00:00 (in 9.999428 seconds) chunkcleanupworker stdout | 2023-08-08 06:36:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:12 UTC)" (scheduled at 2023-08-08 06:36:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:36:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:36:02,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 2, 294939), True, datetime.datetime(2023, 8, 8, 6, 36, 2, 294939), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:36:02,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:36:02,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:36:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:36:02,345 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:36:02,466 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:36:02,859 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:36:03,377 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:36:03,697 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:36:03,849 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:36:03,853 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:36:03,860 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:36:03,863 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:36:03,867 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:36:03,870 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:36:03,883 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:36:03,889 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:36:03,894 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:36:04,121 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} repositorygcworker stdout | 2023-08-08 06:36:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:36:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:09.793658+00:00 (in 5.605742 seconds) repositorygcworker stdout | 2023-08-08 06:36:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:04 UTC)" (scheduled at 2023-08-08 06:36:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:36:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:36:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:04 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:36:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:36:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:15.371682+00:00 (in 9.999511 seconds) notificationworker stdout | 2023-08-08 06:36:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:15 UTC)" (scheduled at 2023-08-08 06:36:05.371682+00:00) notificationworker stdout | 2023-08-08 06:36:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:36:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 5, 372743), True, datetime.datetime(2023, 8, 8, 6, 36, 5, 372743), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:36:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:36:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:36:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:15 UTC)" executed successfully builder stdout | 2023-08-08 06:36:05,942 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:36:05,951 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:36:05,951 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:36:05,955 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:36:07,179 [197] [DEBUG] [app] Starting request: urn:request:345d22ff-3e83-494c-82b0-f96a35ed0a88 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:36:07,180 [198] [DEBUG] [app] Starting request: urn:request:df3e86bc-55ee-45aa-b4b1-3840b67b184b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:36:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:07,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:07,185 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:36:07,185 [209] [DEBUG] [app] Starting request: urn:request:c431e377-fcfb-40d2-ad91-5e0d826e4363 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:36:07,186 [207] [DEBUG] [app] Starting request: urn:request:45120d86-d5a6-4b71-8a52-266492a257fc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:36:07,186 [209] [DEBUG] [app] Ending request: urn:request:c431e377-fcfb-40d2-ad91-5e0d826e4363 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c431e377-fcfb-40d2-ad91-5e0d826e4363', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:36:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:36:07,186 [207] [DEBUG] [app] Ending request: urn:request:45120d86-d5a6-4b71-8a52-266492a257fc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:45120d86-d5a6-4b71-8a52-266492a257fc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 06:36:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:36:07,187 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:36:07,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:07,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:07,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:07,191 [198] [DEBUG] [app] Starting request: urn:request:4eaa32cf-c5fc-4f29-baa0-d7491e905660 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:36:07,192 [198] [DEBUG] [app] Ending request: urn:request:4eaa32cf-c5fc-4f29-baa0-d7491e905660 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4eaa32cf-c5fc-4f29-baa0-d7491e905660', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:36:07,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:36:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:36:07,193 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:07,194 [199] [DEBUG] [app] Starting request: urn:request:5b1216a5-c344-4106-a4ff-d40060562bbb (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:36:07,195 [199] [DEBUG] [app] Ending request: urn:request:5b1216a5-c344-4106-a4ff-d40060562bbb (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5b1216a5-c344-4106-a4ff-d40060562bbb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:36:07,195 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:36:07,195 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:07,196 [198] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:36:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:36:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:37.197412+00:00 (in 29.999573 seconds) buildlogsarchiver stdout | 2023-08-08 06:36:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:37 UTC)" (scheduled at 2023-08-08 06:36:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:36:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 36, 7, 198291), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:36:07,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:36:07,200 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:36:07,204 [197] [DEBUG] [app] Ending request: urn:request:345d22ff-3e83-494c-82b0-f96a35ed0a88 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:345d22ff-3e83-494c-82b0-f96a35ed0a88', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:36:07,204 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:36:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:36:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:36:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:36:07,205 [198] [DEBUG] [app] Ending request: urn:request:df3e86bc-55ee-45aa-b4b1-3840b67b184b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:df3e86bc-55ee-45aa-b4b1-3840b67b184b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} buildlogsarchiver stdout | 2023-08-08 06:36:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:36:07,205 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:36:07,206 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:36:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:36:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:36:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:36:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:36:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:36:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:36:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:36:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:19.417660+00:00 (in 9.999446 seconds) gcworker stdout | 2023-08-08 06:36:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:19 UTC)" (scheduled at 2023-08-08 06:36:09.417660+00:00) gcworker stdout | 2023-08-08 06:36:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266969418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:36:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:36:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:36:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:36:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:36:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:04.187429+00:00 (in 54.393301 seconds) repositorygcworker stdout | 2023-08-08 06:36:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:09 UTC)" (scheduled at 2023-08-08 06:36:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:36:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:36:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 9, 794470), True, datetime.datetime(2023, 8, 8, 6, 36, 9, 794470), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:36:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:36:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:36:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:36:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:36:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:22.293757+00:00 (in 9.999431 seconds) chunkcleanupworker stdout | 2023-08-08 06:36:12,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:22 UTC)" (scheduled at 2023-08-08 06:36:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:36:12,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:36:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 12, 295298), True, datetime.datetime(2023, 8, 8, 6, 36, 12, 295298), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:36:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:36:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:36:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:36:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:36:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:31.283850+00:00 (in 18.185878 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:36:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:13 UTC)" (scheduled at 2023-08-08 06:36:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:36:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:36:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 13, 98488), True, datetime.datetime(2023, 8, 8, 6, 36, 13, 98488), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:36:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:36:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:36:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:36:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:36:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:25.371682+00:00 (in 9.999528 seconds) notificationworker stdout | 2023-08-08 06:36:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:25 UTC)" (scheduled at 2023-08-08 06:36:15.371682+00:00) notificationworker stdout | 2023-08-08 06:36:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:36:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 15, 372482), True, datetime.datetime(2023, 8, 8, 6, 36, 15, 372482), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:36:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:36:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:36:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:25 UTC)" executed successfully builder stdout | 2023-08-08 06:36:15,965 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:36:15,974 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:36:15,974 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:36:15,979 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:36:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:36:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:29.417660+00:00 (in 9.999463 seconds) gcworker stdout | 2023-08-08 06:36:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:29 UTC)" (scheduled at 2023-08-08 06:36:19.417660+00:00) gcworker stdout | 2023-08-08 06:36:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266979418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:36:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:36:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:36:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:29 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:36:19,625 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} securityworker stdout | 2023-08-08 06:36:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:36:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:26.890741+00:00 (in 7.202877 seconds) securityworker stdout | 2023-08-08 06:36:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:49 UTC)" (scheduled at 2023-08-08 06:36:19.687363+00:00) securityworker stdout | 2023-08-08 06:36:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:36:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:36:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:36:19,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:36:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:36:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:36:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 31, 19, 693223), 1, 2]) securityworker stdout | 2023-08-08 06:36:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:19,705 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 31, 19, 693223), 1, 2]) securityworker stdout | 2023-08-08 06:36:19,706 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:36:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:36:20,645 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:36:21,808 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:36:22,178 [197] [DEBUG] [app] Starting request: urn:request:84c30636-0a4a-42ca-87f9-6cad1590f3a7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:36:22,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:22,180 [200] [DEBUG] [app] Starting request: urn:request:84dd69ff-bfaf-4492-8816-7110fde9cde0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:36:22,182 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:36:22,184 [209] [DEBUG] [app] Starting request: urn:request:0c8acc54-312d-47c8-978e-43cffc00b625 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:36:22,185 [208] [DEBUG] [app] Starting request: urn:request:6e30e110-9bb1-489d-8a2a-cd5f15b91d6e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:36:22,185 [209] [DEBUG] [app] Ending request: urn:request:0c8acc54-312d-47c8-978e-43cffc00b625 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0c8acc54-312d-47c8-978e-43cffc00b625', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:36:22,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:36:22,186 [208] [DEBUG] [app] Ending request: urn:request:6e30e110-9bb1-489d-8a2a-cd5f15b91d6e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6e30e110-9bb1-489d-8a2a-cd5f15b91d6e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:36:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:36:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:36:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:22,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:22,191 [199] [DEBUG] [app] Starting request: urn:request:bbabe420-e110-49eb-b4f4-d3e42fc0d3e1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:36:22,191 [198] [DEBUG] [app] Starting request: urn:request:ccae8e5c-75e5-46c5-8870-7fbb801e8181 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:36:22,191 [199] [DEBUG] [app] Ending request: urn:request:bbabe420-e110-49eb-b4f4-d3e42fc0d3e1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bbabe420-e110-49eb-b4f4-d3e42fc0d3e1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:36:22,192 [198] [DEBUG] [app] Ending request: urn:request:ccae8e5c-75e5-46c5-8870-7fbb801e8181 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ccae8e5c-75e5-46c5-8870-7fbb801e8181', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:36:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:36:22,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:36:22,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:36:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:36:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:36:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:36:22,202 [200] [DEBUG] [app] Ending request: urn:request:84dd69ff-bfaf-4492-8816-7110fde9cde0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:84dd69ff-bfaf-4492-8816-7110fde9cde0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:36:22,203 [197] [DEBUG] [app] Ending request: urn:request:84c30636-0a4a-42ca-87f9-6cad1590f3a7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:84c30636-0a4a-42ca-87f9-6cad1590f3a7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:36:22,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:36:22,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:36:22,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:36:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:36:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:36:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:36:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:36:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" storagereplication stdout | 2023-08-08 06:36:22,288 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} chunkcleanupworker stdout | 2023-08-08 06:36:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:36:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:28.874906+00:00 (in 6.580669 seconds) chunkcleanupworker stdout | 2023-08-08 06:36:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:32 UTC)" (scheduled at 2023-08-08 06:36:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:36:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:36:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 22, 294586), True, datetime.datetime(2023, 8, 8, 6, 36, 22, 294586), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:36:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:36:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:36:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:32 UTC)" executed successfully gcworker stdout | 2023-08-08 06:36:24,932 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:36:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:36:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:35.371682+00:00 (in 9.999569 seconds) notificationworker stdout | 2023-08-08 06:36:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:35 UTC)" (scheduled at 2023-08-08 06:36:25.371682+00:00) notificationworker stdout | 2023-08-08 06:36:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:36:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 25, 372435), True, datetime.datetime(2023, 8, 8, 6, 36, 25, 372435), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:36:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:36:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:36:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:35 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:36:25,508 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:36:25,511 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:36:25,514 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} exportactionlogsworker stdout | 2023-08-08 06:36:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:36:25,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:47.398551+00:00 (in 21.883758 seconds) exportactionlogsworker stdout | 2023-08-08 06:36:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:25 UTC)" (scheduled at 2023-08-08 06:36:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:36:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:36:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:25 UTC)" executed successfully builder stdout | 2023-08-08 06:36:25,989 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:36:25,997 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:36:25,997 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:36:26,001 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:36:26,835 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:36:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:36:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:49.687363+00:00 (in 22.796017 seconds) securityworker stdout | 2023-08-08 06:36:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:56 UTC)" (scheduled at 2023-08-08 06:36:26.890741+00:00) securityworker stdout | 2023-08-08 06:36:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:36:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:36:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:36:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:36:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:36:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:36:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:36:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:36:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:36:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:36:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 31, 26, 896697), 1, 2]) securityworker stdout | 2023-08-08 06:36:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 31, 26, 896697), 1, 2]) securityworker stdout | 2023-08-08 06:36:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:36:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:36:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:36:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:36:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:36:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:36:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:36:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:36:56 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:36:27,204 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:36:27,207 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:36:27,902 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:36:28,182 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:36:28,250 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:36:28,294 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:36:28,307 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:36:28,314 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:36:28,726 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} chunkcleanupworker stdout | 2023-08-08 06:36:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:36:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:32.293757+00:00 (in 3.418390 seconds) chunkcleanupworker stdout | 2023-08-08 06:36:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:28 UTC)" (scheduled at 2023-08-08 06:36:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:36:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:36:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:28 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:36:28,949 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:36:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:36:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:39.417660+00:00 (in 9.999482 seconds) gcworker stdout | 2023-08-08 06:36:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:39 UTC)" (scheduled at 2023-08-08 06:36:29.417660+00:00) gcworker stdout | 2023-08-08 06:36:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266989418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:36:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:36:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:36:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:36:30,328 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:36:30,466 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityscanningnotificationworker stdout | 2023-08-08 06:36:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:36:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:54.284093+00:00 (in 22.999680 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:36:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:41:31 UTC)" (scheduled at 2023-08-08 06:36:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:36:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 36, 31, 284880), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:36:31,292 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 36, 31, 284880), True, datetime.datetime(2023, 8, 8, 6, 36, 31, 284880), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:36:31,294 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 36, 31, 284880), True, datetime.datetime(2023, 8, 8, 6, 36, 31, 284880), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 36, 31, 284880), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:36:31,296 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:36:31,296 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:41:31 UTC)" executed successfully securityworker stdout | 2023-08-08 06:36:31,491 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:36:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:36:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:42.293757+00:00 (in 9.999507 seconds) chunkcleanupworker stdout | 2023-08-08 06:36:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:42 UTC)" (scheduled at 2023-08-08 06:36:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:36:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:36:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 32, 294721), True, datetime.datetime(2023, 8, 8, 6, 36, 32, 294721), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:36:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:36:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:36:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:36:32,377 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:36:32,502 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:36:32,875 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:36:33,412 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:36:33,714 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:36:33,862 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:36:33,865 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:36:33,872 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:36:33,875 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:36:33,878 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:36:33,881 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:36:33,893 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:36:33,900 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:36:33,905 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:36:34,158 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:36:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:36:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:45.371682+00:00 (in 9.999558 seconds) notificationworker stdout | 2023-08-08 06:36:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:45 UTC)" (scheduled at 2023-08-08 06:36:35.371682+00:00) notificationworker stdout | 2023-08-08 06:36:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:36:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 35, 372441), True, datetime.datetime(2023, 8, 8, 6, 36, 35, 372441), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:36:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:36:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:36:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:45 UTC)" executed successfully builder stdout | 2023-08-08 06:36:36,002 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:36:36,011 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:36:36,011 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:36:36,016 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:36:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:36:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:54.393627+00:00 (in 17.813280 seconds) namespacegcworker stdout | 2023-08-08 06:36:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:36 UTC)" (scheduled at 2023-08-08 06:36:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:36:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:36:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:36:37,179 [197] [DEBUG] [app] Starting request: urn:request:f18ff996-0e09-4fff-a4f4-45f2a99f1f6d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:36:37,179 [200] [DEBUG] [app] Starting request: urn:request:7c9bb4b2-d034-4d14-bcd1-dd601e266840 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:36:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:36:37,185 [210] [DEBUG] [app] Starting request: urn:request:7b711557-5335-4eb5-a098-b846f4399c91 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:36:37,185 [203] [DEBUG] [app] Starting request: urn:request:74ce31b4-e7a6-4002-b204-48e180bea77f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:36:37,185 [210] [DEBUG] [app] Ending request: urn:request:7b711557-5335-4eb5-a098-b846f4399c91 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7b711557-5335-4eb5-a098-b846f4399c91', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:36:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:36:37,186 [203] [DEBUG] [app] Ending request: urn:request:74ce31b4-e7a6-4002-b204-48e180bea77f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:74ce31b4-e7a6-4002-b204-48e180bea77f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:36:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:36:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:36:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:37,190 [199] [DEBUG] [app] Starting request: urn:request:ae9c42c0-efcd-4855-ab68-f2ecce6ab224 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:36:37,191 [200] [DEBUG] [app] Starting request: urn:request:04715625-ba3b-45f4-a87a-fdc5dbc5b65b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:36:37,191 [199] [DEBUG] [app] Ending request: urn:request:ae9c42c0-efcd-4855-ab68-f2ecce6ab224 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ae9c42c0-efcd-4855-ab68-f2ecce6ab224', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:36:37,191 [200] [DEBUG] [app] Ending request: urn:request:04715625-ba3b-45f4-a87a-fdc5dbc5b65b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:04715625-ba3b-45f4-a87a-fdc5dbc5b65b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:36:37,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:36:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:36:37,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:36:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:36:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:36:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:36:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:36:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:36:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:07.197412+00:00 (in 29.999464 seconds) buildlogsarchiver stdout | 2023-08-08 06:36:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:07 UTC)" (scheduled at 2023-08-08 06:36:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:36:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 36, 37, 198391), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:36:37,203 [197] [DEBUG] [app] Ending request: urn:request:f18ff996-0e09-4fff-a4f4-45f2a99f1f6d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f18ff996-0e09-4fff-a4f4-45f2a99f1f6d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:36:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:36:37,203 [200] [DEBUG] [app] Ending request: urn:request:7c9bb4b2-d034-4d14-bcd1-dd601e266840 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7c9bb4b2-d034-4d14-bcd1-dd601e266840', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:36:37,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:36:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:36:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:36:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:36:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:36:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:36:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:36:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:36:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:36:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:36:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:36:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:36:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:36:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:49.417660+00:00 (in 9.999228 seconds) gcworker stdout | 2023-08-08 06:36:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:49 UTC)" (scheduled at 2023-08-08 06:36:39.417660+00:00) gcworker stdout | 2023-08-08 06:36:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690266999418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:36:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:36:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:36:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:36:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:36:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:52.293757+00:00 (in 9.999516 seconds) chunkcleanupworker stdout | 2023-08-08 06:36:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:52 UTC)" (scheduled at 2023-08-08 06:36:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:36:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:36:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 42, 294714), True, datetime.datetime(2023, 8, 8, 6, 36, 42, 294714), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:36:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:36:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:36:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:36:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:36:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:55.371682+00:00 (in 9.999549 seconds) notificationworker stdout | 2023-08-08 06:36:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:55 UTC)" (scheduled at 2023-08-08 06:36:45.371682+00:00) notificationworker stdout | 2023-08-08 06:36:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:36:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 45, 372469), True, datetime.datetime(2023, 8, 8, 6, 36, 45, 372469), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:36:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:36:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:36:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:55 UTC)" executed successfully builder stdout | 2023-08-08 06:36:46,021 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:36:46,030 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:36:46,030 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:36:46,035 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:36:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:36:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:25.514341+00:00 (in 38.115329 seconds) exportactionlogsworker stdout | 2023-08-08 06:36:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:47 UTC)" (scheduled at 2023-08-08 06:36:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:36:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:36:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 47, 399479), True, datetime.datetime(2023, 8, 8, 6, 36, 47, 399479), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:36:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:36:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:36:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:36:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:36:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:59.417660+00:00 (in 9.999471 seconds) gcworker stdout | 2023-08-08 06:36:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:59 UTC)" (scheduled at 2023-08-08 06:36:49.417660+00:00) gcworker stdout | 2023-08-08 06:36:49,418 [61] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2023-08-08 06:36:49,425 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267009424, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:36:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:36:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:36:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:36:59 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:36:49,656 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} securityworker stdout | 2023-08-08 06:36:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:36:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:36:56.890741+00:00 (in 7.202894 seconds) securityworker stdout | 2023-08-08 06:36:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:19 UTC)" (scheduled at 2023-08-08 06:36:49.687363+00:00) securityworker stdout | 2023-08-08 06:36:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:36:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:36:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:36:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:36:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:36:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:19 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:36:50,665 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:36:51,832 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:36:52,179 [197] [DEBUG] [app] Starting request: urn:request:13e445cc-8436-4611-9c04-9a3e08067b36 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:36:52,179 [198] [DEBUG] [app] Starting request: urn:request:7ba5d3a7-28bc-491c-847d-19fd8fb66349 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:36:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:36:52,185 [210] [DEBUG] [app] Starting request: urn:request:ccace4ca-830e-4456-aa86-d475043d7c52 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:36:52,185 [203] [DEBUG] [app] Starting request: urn:request:f8e6bd83-553d-4526-8c9b-eebd18e02a16 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:36:52,185 [210] [DEBUG] [app] Ending request: urn:request:ccace4ca-830e-4456-aa86-d475043d7c52 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ccace4ca-830e-4456-aa86-d475043d7c52', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:36:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:36:52,186 [203] [DEBUG] [app] Ending request: urn:request:f8e6bd83-553d-4526-8c9b-eebd18e02a16 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f8e6bd83-553d-4526-8c9b-eebd18e02a16', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:36:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:36:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:36:52,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:36:52,191 [198] [DEBUG] [app] Starting request: urn:request:eb44c4b6-7f16-4bc1-ac52-426ad3e9028d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:36:52,191 [198] [DEBUG] [app] Ending request: urn:request:eb44c4b6-7f16-4bc1-ac52-426ad3e9028d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:eb44c4b6-7f16-4bc1-ac52-426ad3e9028d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:36:52,191 [199] [DEBUG] [app] Starting request: urn:request:d833b98f-aa7c-4cf5-b276-13e1ad4ebd42 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:36:52,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:36:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:36:52,192 [199] [DEBUG] [app] Ending request: urn:request:d833b98f-aa7c-4cf5-b276-13e1ad4ebd42 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d833b98f-aa7c-4cf5-b276-13e1ad4ebd42', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:36:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:36:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:36:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:36:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:36:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:36:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:36:52,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:36:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:36:52,202 [198] [DEBUG] [app] Ending request: urn:request:7ba5d3a7-28bc-491c-847d-19fd8fb66349 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7ba5d3a7-28bc-491c-847d-19fd8fb66349', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:36:52,202 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:36:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:36:52,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:36:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:36:52,203 [197] [DEBUG] [app] Ending request: urn:request:13e445cc-8436-4611-9c04-9a3e08067b36 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:13e445cc-8436-4611-9c04-9a3e08067b36', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:36:52,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:36:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:36:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:36:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:36:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:36:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:02.293757+00:00 (in 9.999513 seconds) chunkcleanupworker stdout | 2023-08-08 06:36:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:02 UTC)" (scheduled at 2023-08-08 06:36:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:36:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:36:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 52, 294845), True, datetime.datetime(2023, 8, 8, 6, 36, 52, 294845), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:36:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:36:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:36:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:02 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:36:52,324 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:36:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:36:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:13.097375+00:00 (in 18.812736 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:36:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:54 UTC)" (scheduled at 2023-08-08 06:36:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:36:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:36:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:36:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:36:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:36.579841+00:00 (in 42.185724 seconds) namespacegcworker stdout | 2023-08-08 06:36:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:54 UTC)" (scheduled at 2023-08-08 06:36:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:36:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:36:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 54, 394583), True, datetime.datetime(2023, 8, 8, 6, 36, 54, 394583), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:36:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:36:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:36:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:37:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:36:54,952 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:36:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:36:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:02.178530+00:00 (in 6.806393 seconds) notificationworker stdout | 2023-08-08 06:36:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:05 UTC)" (scheduled at 2023-08-08 06:36:55.371682+00:00) notificationworker stdout | 2023-08-08 06:36:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:36:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 36, 55, 372468), True, datetime.datetime(2023, 8, 8, 6, 36, 55, 372468), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:36:55,379 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:36:55,379 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:36:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:36:55,517 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:36:55,520 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:36:55,525 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:36:56,038 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:36:56,046 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:36:56,046 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:36:56,051 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:36:56,867 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:36:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:36:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:19.687363+00:00 (in 22.796050 seconds) securityworker stdout | 2023-08-08 06:36:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:26 UTC)" (scheduled at 2023-08-08 06:36:56.890741+00:00) securityworker stdout | 2023-08-08 06:36:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:36:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:36:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:36:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:36:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:36:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:36:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:36:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:36:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:36:56,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:36:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 31, 56, 896548), 1, 2]) securityworker stdout | 2023-08-08 06:36:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 31, 56, 896548), 1, 2]) securityworker stdout | 2023-08-08 06:36:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:36:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:36:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:36:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:36:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:36:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:36:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:36:56,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:36:56,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:36:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:26 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:36:57,218 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:36:57,244 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:36:57,910 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:36:58,195 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:36:58,281 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:36:58,307 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:36:58,320 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:36:58,325 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:36:58,741 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:36:58,986 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:36:59,417 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:36:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:09.417660+00:00 (in 9.999542 seconds) gcworker stdout | 2023-08-08 06:36:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:09 UTC)" (scheduled at 2023-08-08 06:36:59.417660+00:00) gcworker stdout | 2023-08-08 06:36:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267019418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:36:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:36:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:36:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:37:00,348 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:37:00,503 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:37:01,506 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:37:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:37:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:05.371682+00:00 (in 3.192678 seconds) notificationworker stdout | 2023-08-08 06:37:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:02 UTC)" (scheduled at 2023-08-08 06:37:02.178530+00:00) notificationworker stdout | 2023-08-08 06:37:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:37:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:37:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:37:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:12.293757+00:00 (in 9.999442 seconds) chunkcleanupworker stdout | 2023-08-08 06:37:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:12 UTC)" (scheduled at 2023-08-08 06:37:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:37:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:37:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 2, 294873), True, datetime.datetime(2023, 8, 8, 6, 37, 2, 294873), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:37:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:37:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:37:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:37:02,415 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:37:02,540 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:37:02,889 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:37:03,435 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:37:03,737 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:37:03,875 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:37:03,878 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:37:03,882 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:37:03,886 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:37:03,889 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:37:03,892 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:37:03,900 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:37:03,910 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:37:03,915 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:37:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:37:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:09.793658+00:00 (in 5.605723 seconds) repositorygcworker stdout | 2023-08-08 06:37:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:04 UTC)" (scheduled at 2023-08-08 06:37:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:37:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:37:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:04 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:37:04,195 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:37:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:37:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:15.371682+00:00 (in 9.999541 seconds) notificationworker stdout | 2023-08-08 06:37:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:15 UTC)" (scheduled at 2023-08-08 06:37:05.371682+00:00) notificationworker stdout | 2023-08-08 06:37:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:37:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 5, 372494), True, datetime.datetime(2023, 8, 8, 6, 37, 5, 372494), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:37:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:37:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:37:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:15 UTC)" executed successfully builder stdout | 2023-08-08 06:37:06,056 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:37:06,064 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:37:06,064 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:37:06,069 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:37:07,179 [197] [DEBUG] [app] Starting request: urn:request:c585fb87-9e39-46cd-86d4-8451ddde643f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:37:07,180 [199] [DEBUG] [app] Starting request: urn:request:e8ab5fdf-d8ed-4d3a-ac7f-19ce249050c3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:37:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:07,182 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:37:07,184 [208] [DEBUG] [app] Starting request: urn:request:c3199332-a9b1-49eb-9435-daeb87c41ece (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:07,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:37:07,185 [208] [DEBUG] [app] Ending request: urn:request:c3199332-a9b1-49eb-9435-daeb87c41ece (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c3199332-a9b1-49eb-9435-daeb87c41ece', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:37:07,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:37:07,185 [210] [DEBUG] [app] Starting request: urn:request:e619e7a1-4a50-4be2-b5dd-6953465ca3da (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:37:07,186 [210] [DEBUG] [app] Ending request: urn:request:e619e7a1-4a50-4be2-b5dd-6953465ca3da (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e619e7a1-4a50-4be2-b5dd-6953465ca3da', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:37:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:37:07,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:07,191 [200] [DEBUG] [app] Starting request: urn:request:05c757fe-ecc0-4840-be52-deb2035b58de (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:07,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:07,191 [200] [DEBUG] [app] Ending request: urn:request:05c757fe-ecc0-4840-be52-deb2035b58de (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:05c757fe-ecc0-4840-be52-deb2035b58de', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:37:07,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:37:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:07,192 [198] [DEBUG] [app] Starting request: urn:request:e4fdfa4f-e680-40a7-97b8-8c6395527b4d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:37:07,192 [198] [DEBUG] [app] Ending request: urn:request:e4fdfa4f-e680-40a7-97b8-8c6395527b4d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e4fdfa4f-e680-40a7-97b8-8c6395527b4d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:37:07,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:37:07,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:07,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:37:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:37:07,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:37:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:37:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:37.197412+00:00 (in 29.999492 seconds) buildlogsarchiver stdout | 2023-08-08 06:37:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:37 UTC)" (scheduled at 2023-08-08 06:37:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:37:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 37, 7, 198329), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:37:07,202 [197] [DEBUG] [app] Ending request: urn:request:c585fb87-9e39-46cd-86d4-8451ddde643f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c585fb87-9e39-46cd-86d4-8451ddde643f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:37:07,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:37:07,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:37:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:37:07,202 [199] [DEBUG] [app] Ending request: urn:request:e8ab5fdf-d8ed-4d3a-ac7f-19ce249050c3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e8ab5fdf-d8ed-4d3a-ac7f-19ce249050c3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:37:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:37:07,202 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:37:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:37:07,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:37:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:37:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:37:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:37:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:37:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:37:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:37:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:37:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:19.417660+00:00 (in 9.999491 seconds) gcworker stdout | 2023-08-08 06:37:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:19 UTC)" (scheduled at 2023-08-08 06:37:09.417660+00:00) gcworker stdout | 2023-08-08 06:37:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267029418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:37:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:37:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:37:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:37:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:37:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:04.187429+00:00 (in 54.393284 seconds) repositorygcworker stdout | 2023-08-08 06:37:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:09 UTC)" (scheduled at 2023-08-08 06:37:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:37:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:37:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 9, 794484), True, datetime.datetime(2023, 8, 8, 6, 37, 9, 794484), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:37:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:37:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:37:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:37:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:37:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:22.293757+00:00 (in 9.999445 seconds) chunkcleanupworker stdout | 2023-08-08 06:37:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:22 UTC)" (scheduled at 2023-08-08 06:37:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:37:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:37:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 12, 294843), True, datetime.datetime(2023, 8, 8, 6, 37, 12, 294843), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:37:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:37:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:37:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:37:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:37:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:54.284093+00:00 (in 41.186194 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:37:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:13 UTC)" (scheduled at 2023-08-08 06:37:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:37:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:37:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 13, 98536), True, datetime.datetime(2023, 8, 8, 6, 37, 13, 98536), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:37:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:37:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:37:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:37:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:37:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:25.371682+00:00 (in 9.999506 seconds) notificationworker stdout | 2023-08-08 06:37:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:25 UTC)" (scheduled at 2023-08-08 06:37:15.371682+00:00) notificationworker stdout | 2023-08-08 06:37:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:37:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 15, 372535), True, datetime.datetime(2023, 8, 8, 6, 37, 15, 372535), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:37:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:37:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:37:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:25 UTC)" executed successfully builder stdout | 2023-08-08 06:37:16,079 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:37:16,088 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:37:16,088 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:37:16,092 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:37:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:37:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:29.417660+00:00 (in 9.999467 seconds) gcworker stdout | 2023-08-08 06:37:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:29 UTC)" (scheduled at 2023-08-08 06:37:19.417660+00:00) gcworker stdout | 2023-08-08 06:37:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267039418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:37:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:37:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:37:19,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:37:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:37:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:26.890741+00:00 (in 7.202841 seconds) securityworker stdout | 2023-08-08 06:37:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:49 UTC)" (scheduled at 2023-08-08 06:37:19.687363+00:00) securityworker stdout | 2023-08-08 06:37:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:37:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:37:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:37:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) notificationworker stdout | 2023-08-08 06:37:19,693 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} securityworker stdout | 2023-08-08 06:37:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:37:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:37:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 32, 19, 691827), 1, 2]) securityworker stdout | 2023-08-08 06:37:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 32, 19, 691827), 1, 2]) securityworker stdout | 2023-08-08 06:37:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:37:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:49 UTC)" executed successfully logrotateworker stdout | 2023-08-08 06:37:20,681 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:37:21,850 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:37:22,179 [200] [DEBUG] [app] Starting request: urn:request:a60cc83c-5a75-468e-a0f2-1f73b651383c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:37:22,179 [199] [DEBUG] [app] Starting request: urn:request:45631a82-725d-4bc1-a094-23482b4551a5 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:37:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:37:22,184 [203] [DEBUG] [app] Starting request: urn:request:90569303-96d9-4b6f-a6a7-484df4c3b1cb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:37:22,185 [203] [DEBUG] [app] Ending request: urn:request:90569303-96d9-4b6f-a6a7-484df4c3b1cb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:90569303-96d9-4b6f-a6a7-484df4c3b1cb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:37:22,185 [210] [DEBUG] [app] Starting request: urn:request:2fb6775f-7798-473c-a132-c291c333e8ae (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:37:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:37:22,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:37:22,186 [210] [DEBUG] [app] Ending request: urn:request:2fb6775f-7798-473c-a132-c291c333e8ae (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2fb6775f-7798-473c-a132-c291c333e8ae', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:37:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:37:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:22,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:22,190 [200] [DEBUG] [app] Starting request: urn:request:3fabdb75-f154-4064-9cff-27d868a76e7d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:22,190 [200] [DEBUG] [app] Ending request: urn:request:3fabdb75-f154-4064-9cff-27d868a76e7d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3fabdb75-f154-4064-9cff-27d868a76e7d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:37:22,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:37:22,191 [197] [DEBUG] [app] Starting request: urn:request:d23153c0-c05f-4f6f-a489-5404ae2f4207 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:22,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:22,191 [197] [DEBUG] [app] Ending request: urn:request:d23153c0-c05f-4f6f-a489-5404ae2f4207 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d23153c0-c05f-4f6f-a489-5404ae2f4207', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:37:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:37:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:37:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:37:22,230 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:37:22,240 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:37:22,256 [199] [DEBUG] [app] Ending request: urn:request:45631a82-725d-4bc1-a094-23482b4551a5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:45631a82-725d-4bc1-a094-23482b4551a5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:37:22,256 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:37:22,256 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:37:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:37:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.078 47 0.078) gunicorn-web stdout | 2023-08-08 06:37:22,259 [200] [DEBUG] [app] Ending request: urn:request:a60cc83c-5a75-468e-a0f2-1f73b651383c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a60cc83c-5a75-468e-a0f2-1f73b651383c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:37:22,260 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:37:22,260 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:37:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:37:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.082 47 0.082) chunkcleanupworker stdout | 2023-08-08 06:37:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:37:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:28.874906+00:00 (in 6.580485 seconds) chunkcleanupworker stdout | 2023-08-08 06:37:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:32 UTC)" (scheduled at 2023-08-08 06:37:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:37:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:37:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 22, 294981), True, datetime.datetime(2023, 8, 8, 6, 37, 22, 294981), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:37:22,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:37:22,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:37:22,305 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:32 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:37:22,361 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gcworker stdout | 2023-08-08 06:37:24,981 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:37:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:37:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:35.371682+00:00 (in 9.999479 seconds) notificationworker stdout | 2023-08-08 06:37:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:35 UTC)" (scheduled at 2023-08-08 06:37:25.371682+00:00) notificationworker stdout | 2023-08-08 06:37:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:37:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 25, 372662), True, datetime.datetime(2023, 8, 8, 6, 37, 25, 372662), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:37:25,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:37:25,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:37:25,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:37:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:37:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:47.398551+00:00 (in 21.883689 seconds) exportactionlogsworker stdout | 2023-08-08 06:37:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:25 UTC)" (scheduled at 2023-08-08 06:37:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:37:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:37:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:25 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:37:25,525 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:37:25,528 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:37:25,534 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:37:26,102 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:37:26,111 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:37:26,111 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:37:26,115 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:37:26,889 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} securityworker stdout | 2023-08-08 06:37:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:37:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:49.687363+00:00 (in 22.796135 seconds) securityworker stdout | 2023-08-08 06:37:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:56 UTC)" (scheduled at 2023-08-08 06:37:26.890741+00:00) securityworker stdout | 2023-08-08 06:37:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:37:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:37:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:37:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:37:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:37:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:37:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:37:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:37:26,898 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:37:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:37:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 32, 26, 897837), 1, 2]) securityworker stdout | 2023-08-08 06:37:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:26,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 32, 26, 897837), 1, 2]) securityworker stdout | 2023-08-08 06:37:26,910 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:26,910 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:26,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:37:26,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:37:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:37:26,911 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:37:26,911 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:37:26,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:37:26,911 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:37:26,911 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:37:56 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:37:27,256 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} chunkcleanupworker stdout | 2023-08-08 06:37:27,259 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} gunicorn-web stdout | 2023-08-08 06:37:27,920 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:37:28,207 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:37:28,305 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:37:28,321 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:37:28,334 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:37:28,337 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:37:28,772 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} chunkcleanupworker stdout | 2023-08-08 06:37:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:37:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:32.293757+00:00 (in 3.418317 seconds) chunkcleanupworker stdout | 2023-08-08 06:37:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:28 UTC)" (scheduled at 2023-08-08 06:37:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:37:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:37:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:28 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:37:29,010 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:37:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:37:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:39.417660+00:00 (in 9.999490 seconds) gcworker stdout | 2023-08-08 06:37:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:39 UTC)" (scheduled at 2023-08-08 06:37:29.417660+00:00) gcworker stdout | 2023-08-08 06:37:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267049418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:37:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:37:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:37:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:37:30,373 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:37:30,527 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:37:31,536 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:37:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:37:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:42.293757+00:00 (in 9.999505 seconds) chunkcleanupworker stdout | 2023-08-08 06:37:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:42 UTC)" (scheduled at 2023-08-08 06:37:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:37:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:37:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 32, 294772), True, datetime.datetime(2023, 8, 8, 6, 37, 32, 294772), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:37:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:37:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:37:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:37:32,422 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:37:32,562 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:37:32,927 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:37:33,474 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:37:33,773 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:37:33,886 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:37:33,889 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:37:33,892 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:37:33,895 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:37:33,898 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:37:33,903 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:37:33,911 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:37:33,918 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:37:33,924 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:37:34,233 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:37:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:37:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:45.371682+00:00 (in 9.999458 seconds) notificationworker stdout | 2023-08-08 06:37:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:45 UTC)" (scheduled at 2023-08-08 06:37:35.371682+00:00) notificationworker stdout | 2023-08-08 06:37:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:37:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 35, 372575), True, datetime.datetime(2023, 8, 8, 6, 37, 35, 372575), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:37:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:37:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:37:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:45 UTC)" executed successfully builder stdout | 2023-08-08 06:37:36,125 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:37:36,134 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:37:36,134 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:37:36,138 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:37:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:37:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:54.393627+00:00 (in 17.813269 seconds) namespacegcworker stdout | 2023-08-08 06:37:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:36 UTC)" (scheduled at 2023-08-08 06:37:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:37:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:37:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:37:37,180 [197] [DEBUG] [app] Starting request: urn:request:5c4a4f76-bdb4-4217-8936-7861f3674c08 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:37:37,180 [200] [DEBUG] [app] Starting request: urn:request:3d159819-cdab-4b39-a183-ed8aa3240bf2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:37:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:37,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:37,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:37,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:37:37,185 [209] [DEBUG] [app] Starting request: urn:request:621ea684-899e-40a4-a406-2219ec39ef0d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:37:37,186 [209] [DEBUG] [app] Ending request: urn:request:621ea684-899e-40a4-a406-2219ec39ef0d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:621ea684-899e-40a4-a406-2219ec39ef0d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:37:37,186 [210] [DEBUG] [app] Starting request: urn:request:f307dfdb-260a-4638-8c35-b7adc36fa9c0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:37:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:37:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:37:37,187 [210] [DEBUG] [app] Ending request: urn:request:f307dfdb-260a-4638-8c35-b7adc36fa9c0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f307dfdb-260a-4638-8c35-b7adc36fa9c0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:37:37,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2023-08-08 06:37:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:37,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:37,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:37,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:37,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:37,193 [198] [DEBUG] [app] Starting request: urn:request:65f603fa-c388-405c-adbf-91f28a68b7aa (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:37,193 [198] [DEBUG] [app] Ending request: urn:request:65f603fa-c388-405c-adbf-91f28a68b7aa (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:65f603fa-c388-405c-adbf-91f28a68b7aa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:37:37,193 [199] [DEBUG] [app] Starting request: urn:request:62afde55-accf-4c14-bb2a-8b81fdbcb166 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:37:37,193 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:37:37,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:37,194 [199] [DEBUG] [app] Ending request: urn:request:62afde55-accf-4c14-bb2a-8b81fdbcb166 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:62afde55-accf-4c14-bb2a-8b81fdbcb166', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:37:37,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:37:37,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:37,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:37:37,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:37:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:37:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:07.197412+00:00 (in 29.999526 seconds) buildlogsarchiver stdout | 2023-08-08 06:37:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:07 UTC)" (scheduled at 2023-08-08 06:37:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:37:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 37, 37, 198294), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:37:37,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:37:37,201 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:37:37,205 [200] [DEBUG] [app] Ending request: urn:request:3d159819-cdab-4b39-a183-ed8aa3240bf2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3d159819-cdab-4b39-a183-ed8aa3240bf2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:37:37,205 [200] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:37:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:37:37,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:37:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:37:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:37:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 gunicorn-web stdout | 2023-08-08 06:37:37,206 [197] [DEBUG] [app] Ending request: urn:request:5c4a4f76-bdb4-4217-8936-7861f3674c08 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5c4a4f76-bdb4-4217-8936-7861f3674c08', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:37:37,207 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:37:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.028) gunicorn-web stdout | 2023-08-08 06:37:37,207 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:37:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:37:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:37:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:37:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:37:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:37:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:49.417660+00:00 (in 9.999488 seconds) gcworker stdout | 2023-08-08 06:37:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:49 UTC)" (scheduled at 2023-08-08 06:37:39.417660+00:00) gcworker stdout | 2023-08-08 06:37:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267059418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:37:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:37:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:37:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:37:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:37:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:52.293757+00:00 (in 9.999529 seconds) chunkcleanupworker stdout | 2023-08-08 06:37:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:52 UTC)" (scheduled at 2023-08-08 06:37:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:37:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:37:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 42, 294712), True, datetime.datetime(2023, 8, 8, 6, 37, 42, 294712), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:37:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:37:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:37:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:37:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:37:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:55.371682+00:00 (in 9.999533 seconds) notificationworker stdout | 2023-08-08 06:37:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:55 UTC)" (scheduled at 2023-08-08 06:37:45.371682+00:00) notificationworker stdout | 2023-08-08 06:37:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:37:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 45, 372779), True, datetime.datetime(2023, 8, 8, 6, 37, 45, 372779), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:37:45,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:37:45,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:37:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:55 UTC)" executed successfully builder stdout | 2023-08-08 06:37:46,148 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:37:46,158 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:37:46,158 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:37:46,162 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:37:47,399 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:37:47,400 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:25.514341+00:00 (in 38.114415 seconds) exportactionlogsworker stdout | 2023-08-08 06:37:47,400 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:47 UTC)" (scheduled at 2023-08-08 06:37:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:37:47,400 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:37:47,401 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 47, 400593), True, datetime.datetime(2023, 8, 8, 6, 37, 47, 400593), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:37:47,410 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:37:47,410 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:37:47,410 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:37:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:37:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:59.417660+00:00 (in 9.999513 seconds) gcworker stdout | 2023-08-08 06:37:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:59 UTC)" (scheduled at 2023-08-08 06:37:49.417660+00:00) gcworker stdout | 2023-08-08 06:37:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267069418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:37:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:37:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:37:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:37:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:37:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:37:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:37:56.890741+00:00 (in 7.202819 seconds) securityworker stdout | 2023-08-08 06:37:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:19 UTC)" (scheduled at 2023-08-08 06:37:49.687363+00:00) securityworker stdout | 2023-08-08 06:37:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:37:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:37:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:37:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:37:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:37:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:37:49,730 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:37:50,718 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:37:51,888 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:37:52,178 [197] [DEBUG] [app] Starting request: urn:request:33c76d02-61d1-4b5f-bb62-e9e58c5de30f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:37:52,179 [200] [DEBUG] [app] Starting request: urn:request:5abc7773-c11a-45f0-9027-7016c0185096 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:37:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:37:52,184 [210] [DEBUG] [app] Starting request: urn:request:fe30cd18-3dda-4622-a421-5170c44175ef (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:37:52,185 [207] [DEBUG] [app] Starting request: urn:request:74e73848-2f0a-4e67-a60d-132cece258ce (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:37:52,185 [210] [DEBUG] [app] Ending request: urn:request:fe30cd18-3dda-4622-a421-5170c44175ef (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fe30cd18-3dda-4622-a421-5170c44175ef', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:37:52,185 [207] [DEBUG] [app] Ending request: urn:request:74e73848-2f0a-4e67-a60d-132cece258ce (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:74e73848-2f0a-4e67-a60d-132cece258ce', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:37:52,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:37:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:37:52,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:37:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:37:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:37:52,190 [199] [DEBUG] [app] Starting request: urn:request:136c6b3d-f762-4c2d-8c82-1cb0f459910a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:52,191 [197] [DEBUG] [app] Starting request: urn:request:96317011-cc8d-4cc5-bf8b-d1574f2e8eb7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:37:52,191 [199] [DEBUG] [app] Ending request: urn:request:136c6b3d-f762-4c2d-8c82-1cb0f459910a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:136c6b3d-f762-4c2d-8c82-1cb0f459910a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:37:52,191 [197] [DEBUG] [app] Ending request: urn:request:96317011-cc8d-4cc5-bf8b-d1574f2e8eb7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:96317011-cc8d-4cc5-bf8b-d1574f2e8eb7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:37:52,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:37:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:37:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:37:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:37:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:37:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:37:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:37:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:37:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:37:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:37:52,203 [200] [DEBUG] [app] Ending request: urn:request:5abc7773-c11a-45f0-9027-7016c0185096 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5abc7773-c11a-45f0-9027-7016c0185096', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:37:52,203 [197] [DEBUG] [app] Ending request: urn:request:33c76d02-61d1-4b5f-bb62-e9e58c5de30f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:33c76d02-61d1-4b5f-bb62-e9e58c5de30f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:37:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:37:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:37:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:37:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:37:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:37:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:37:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:37:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:37:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:37:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:02.293757+00:00 (in 9.999489 seconds) chunkcleanupworker stdout | 2023-08-08 06:37:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:02 UTC)" (scheduled at 2023-08-08 06:37:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:37:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:37:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 52, 294777), True, datetime.datetime(2023, 8, 8, 6, 37, 52, 294777), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:37:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:37:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:37:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:02 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:37:52,382 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:37:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:37:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:13.097375+00:00 (in 18.812671 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:37:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:54 UTC)" (scheduled at 2023-08-08 06:37:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:37:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:37:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:37:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:37:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:36.579841+00:00 (in 42.185639 seconds) namespacegcworker stdout | 2023-08-08 06:37:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:54 UTC)" (scheduled at 2023-08-08 06:37:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:37:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:37:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 54, 394858), True, datetime.datetime(2023, 8, 8, 6, 37, 54, 394858), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:37:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:37:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:37:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:38:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:37:55,019 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:37:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:37:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:02.178530+00:00 (in 6.806376 seconds) notificationworker stdout | 2023-08-08 06:37:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:05 UTC)" (scheduled at 2023-08-08 06:37:55.371682+00:00) notificationworker stdout | 2023-08-08 06:37:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:37:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 37, 55, 372597), True, datetime.datetime(2023, 8, 8, 6, 37, 55, 372597), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:37:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:37:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:37:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:37:55,534 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:37:55,537 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:37:55,541 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:37:56,170 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:37:56,179 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:37:56,179 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:37:56,183 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:37:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:37:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:19.687363+00:00 (in 22.796079 seconds) securityworker stdout | 2023-08-08 06:37:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:26 UTC)" (scheduled at 2023-08-08 06:37:56.890741+00:00) securityworker stdout | 2023-08-08 06:37:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:37:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:37:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:37:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:37:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:37:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:37:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:37:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:37:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:37:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:37:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 32, 56, 896809), 1, 2]) securityworker stdout | 2023-08-08 06:37:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 32, 56, 896809), 1, 2]) securityworker stdout | 2023-08-08 06:37:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:37:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:37:56,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:37:56,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:37:56,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:37:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:37:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:37:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:37:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:37:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:26 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:37:56,925 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:37:57,280 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:37:57,284 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:37:57,929 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:37:58,220 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} teamsyncworker stdout | 2023-08-08 06:37:58,329 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:37:58,335 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:37:58,347 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:37:58,350 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} repositoryactioncounter stdout | 2023-08-08 06:37:58,811 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:37:59,033 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:37:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:37:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:09.417660+00:00 (in 9.999511 seconds) gcworker stdout | 2023-08-08 06:37:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:09 UTC)" (scheduled at 2023-08-08 06:37:59.417660+00:00) gcworker stdout | 2023-08-08 06:37:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267079418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:37:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:37:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:37:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:38:00,410 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:38:00,564 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:38:01,548 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:38:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:38:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:05.371682+00:00 (in 3.192669 seconds) notificationworker stdout | 2023-08-08 06:38:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:02 UTC)" (scheduled at 2023-08-08 06:38:02.178530+00:00) notificationworker stdout | 2023-08-08 06:38:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:38:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:38:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:38:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:12.293757+00:00 (in 9.999501 seconds) chunkcleanupworker stdout | 2023-08-08 06:38:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:12 UTC)" (scheduled at 2023-08-08 06:38:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:38:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:38:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 2, 294756), True, datetime.datetime(2023, 8, 8, 6, 38, 2, 294756), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:38:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:38:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:38:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:38:02,460 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:38:02,581 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:38:02,943 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:38:03,492 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:38:03,788 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:38:03,898 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:38:03,902 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:38:03,907 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:38:03,909 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:38:03,912 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:38:03,915 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:38:03,922 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:38:03,927 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:38:03,933 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:38:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:38:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:09.793658+00:00 (in 5.605759 seconds) repositorygcworker stdout | 2023-08-08 06:38:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:04 UTC)" (scheduled at 2023-08-08 06:38:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:38:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:38:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:04 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:38:04,271 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:38:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:38:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:15.371682+00:00 (in 9.999536 seconds) notificationworker stdout | 2023-08-08 06:38:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:15 UTC)" (scheduled at 2023-08-08 06:38:05.371682+00:00) notificationworker stdout | 2023-08-08 06:38:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:38:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 5, 372738), True, datetime.datetime(2023, 8, 8, 6, 38, 5, 372738), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:38:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:38:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:38:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:15 UTC)" executed successfully builder stdout | 2023-08-08 06:38:06,194 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:38:06,203 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:38:06,203 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:38:06,207 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:07,179 [200] [DEBUG] [app] Starting request: urn:request:5aa3048e-5515-44e4-858a-3c36655f33ab (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:38:07,179 [199] [DEBUG] [app] Starting request: urn:request:c68ae1ef-7337-481f-adc3-2be18c244b59 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:38:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:07,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:38:07,185 [210] [DEBUG] [app] Starting request: urn:request:85f9a720-dc87-40a1-9aa4-e1de0e3055b6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:38:07,185 [208] [DEBUG] [app] Starting request: urn:request:45727a9e-784c-4cac-b135-3ba1e47d049c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:38:07,185 [210] [DEBUG] [app] Ending request: urn:request:85f9a720-dc87-40a1-9aa4-e1de0e3055b6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:85f9a720-dc87-40a1-9aa4-e1de0e3055b6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:38:07,186 [208] [DEBUG] [app] Ending request: urn:request:45727a9e-784c-4cac-b135-3ba1e47d049c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:45727a9e-784c-4cac-b135-3ba1e47d049c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:38:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:38:07,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:38:07,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:07,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:07,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:07,191 [197] [DEBUG] [app] Starting request: urn:request:ccaa9076-3069-4836-b441-f82519b444b2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:38:07,192 [197] [DEBUG] [app] Ending request: urn:request:ccaa9076-3069-4836-b441-f82519b444b2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ccaa9076-3069-4836-b441-f82519b444b2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:38:07,192 [200] [DEBUG] [app] Starting request: urn:request:b6d94c29-5c83-4f41-9638-bb02e6e34ff8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:38:07,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:38:07,192 [200] [DEBUG] [app] Ending request: urn:request:b6d94c29-5c83-4f41-9638-bb02e6e34ff8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b6d94c29-5c83-4f41-9638-bb02e6e34ff8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:38:07,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:38:07,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:07,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:07,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:38:07,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:38:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:38:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:37.197412+00:00 (in 29.999561 seconds) buildlogsarchiver stdout | 2023-08-08 06:38:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:37 UTC)" (scheduled at 2023-08-08 06:38:07.197412+00:00) gunicorn-web stdout | 2023-08-08 06:38:07,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:38:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 38, 7, 198253), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:38:07,204 [199] [DEBUG] [app] Ending request: urn:request:c68ae1ef-7337-481f-adc3-2be18c244b59 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c68ae1ef-7337-481f-adc3-2be18c244b59', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:38:07,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:07,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:38:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:38:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) gunicorn-web stdout | 2023-08-08 06:38:07,205 [200] [DEBUG] [app] Ending request: urn:request:5aa3048e-5515-44e4-858a-3c36655f33ab (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5aa3048e-5515-44e4-858a-3c36655f33ab', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:38:07,205 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:07,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:38:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:38:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.028) buildlogsarchiver stdout | 2023-08-08 06:38:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:38:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:38:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:38:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:38:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:38:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:38:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:19.417660+00:00 (in 9.999497 seconds) gcworker stdout | 2023-08-08 06:38:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:19 UTC)" (scheduled at 2023-08-08 06:38:09.417660+00:00) gcworker stdout | 2023-08-08 06:38:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267089418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:38:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:38:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:38:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:38:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:38:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:04.187429+00:00 (in 54.393237 seconds) repositorygcworker stdout | 2023-08-08 06:38:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:09 UTC)" (scheduled at 2023-08-08 06:38:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:38:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:38:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 9, 794544), True, datetime.datetime(2023, 8, 8, 6, 38, 9, 794544), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:38:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:38:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:38:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:38:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:38:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:22.293757+00:00 (in 9.999479 seconds) chunkcleanupworker stdout | 2023-08-08 06:38:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:22 UTC)" (scheduled at 2023-08-08 06:38:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:38:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:38:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 12, 294776), True, datetime.datetime(2023, 8, 8, 6, 38, 12, 294776), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:38:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:38:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:38:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:38:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:38:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:54.284093+00:00 (in 41.186187 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:38:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:13 UTC)" (scheduled at 2023-08-08 06:38:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:38:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:38:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 13, 98386), True, datetime.datetime(2023, 8, 8, 6, 38, 13, 98386), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:38:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:38:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:38:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:13 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:14,595 [199] [DEBUG] [app] Starting request: urn:request:5a45b97d-2f5c-4892-949b-31b60fd769b1 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:14,595 [200] [DEBUG] [app] Starting request: urn:request:6cc1a4f3-213d-428b-b662-82b55e7293c9 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:14,595 [198] [DEBUG] [app] Starting request: urn:request:dc33c53e-6291-4f77-9612-0e33afe59712 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:14,595 [197] [DEBUG] [app] Starting request: urn:request:bb614777-8eed-4fcd-9ecb-267c6fffd0dc (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:14,596 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,596 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,596 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,596 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,596 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,596 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,596 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,596 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,596 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,596 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,596 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,596 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,603 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:14,603 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:14,603 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,603 [200] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:38:14,603 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,603 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,603 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,603 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,604 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:14,604 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:14,604 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:14,604 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:14,604 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,604 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,604 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:14,605 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:14,605 [197] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:38:14,605 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,605 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,605 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,605 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,605 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,605 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,605 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,605 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:38:14,605 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,605 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,605 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 50, 14, 605363), datetime.datetime(2023, 8, 8, 6, 38, 14, 605413)]) gunicorn-web stdout | 2023-08-08 06:38:14,606 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:38:14,606 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,607 [197] [DEBUG] [app] Ending request: urn:request:bb614777-8eed-4fcd-9ecb-267c6fffd0dc (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:bb614777-8eed-4fcd-9ecb-267c6fffd0dc', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:14,608 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.014 62 0.014) gunicorn-web stdout | 2023-08-08 06:38:14,608 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:14,608 [198] [DEBUG] [peewee] ('((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t3"."user_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'user1', 3, 3, 1, 'user1', 3, 3, 1, 'user1', 1, 3, 101]) gunicorn-web stdout | 2023-08-08 06:38:14,609 [197] [DEBUG] [app] Starting request: urn:request:654c8e92-9054-4b47-bea1-8f242b943309 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:14,609 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,609 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,609 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:38:14,609 [200] [DEBUG] [app] Ending request: urn:request:6cc1a4f3-213d-428b-b662-82b55e7293c9 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:6cc1a4f3-213d-428b-b662-82b55e7293c9', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:14,610 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,610 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 878 0.017) gunicorn-web stdout | 2023-08-08 06:38:14,611 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/github/trigger/58f84ee1-b064-4be3-aeb1-b8299deea05c" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:14,611 [199] [DEBUG] [app] Ending request: urn:request:5a45b97d-2f5c-4892-949b-31b60fd769b1 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:5a45b97d-2f5c-4892-949b-31b60fd769b1', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:14,612 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.018 156 0.018) gunicorn-web stdout | 2023-08-08 06:38:14,612 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:14,614 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:38:14,616 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:14,616 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:14,616 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,616 [198] [DEBUG] [app] Ending request: urn:request:dc33c53e-6291-4f77-9612-0e33afe59712 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:dc33c53e-6291-4f77-9612-0e33afe59712', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'user1', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:14,616 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:14,616 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,616 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,616 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:14,617 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:14,617 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,617 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1&popularity=true"a=true&state=true HTTP/2.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.023 101 0.023) gunicorn-web stdout | 2023-08-08 06:38:14,620 [197] [DEBUG] [peewee] ('((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t3"."user_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'user1_build_test', 3, 3, 1, 'user1_build_test', 3, 3, 1, 'user1_build_test', 1, 3, 101]) gunicorn-web stdout | 2023-08-08 06:38:14,625 [197] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Max("t1"."lifetime_start_ms") FROM "tag" AS "t1" WHERE ((("t1"."repository_id" IN (%s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) GROUP BY "t1"."repository_id"', [4, None, 1691476694625, False]) gunicorn-web stdout | 2023-08-08 06:38:14,627 [197] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Sum("t1"."count") FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" IN (%s)) AND ("t1"."date" >= %s)) GROUP BY "t1"."repository_id"', [4, datetime.date(2023, 8, 1)]) gunicorn-web stdout | 2023-08-08 06:38:14,629 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:38:14,631 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,633 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:38:14,634 [197] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,636 [197] [DEBUG] [peewee] ('SELECT Sum("t1"."layers_compressed_size") AS "size_bytes" FROM "manifest" AS "t1" INNER JOIN (SELECT "t2"."manifest_id" FROM "tag" AS "t2" WHERE ((("t2"."hidden" = %s) AND (("t2"."lifetime_end_ms" IS %s) OR ("t2"."lifetime_end_ms" > %s))) AND ("t2"."repository_id" = %s)) GROUP BY "t2"."manifest_id" HAVING (Count("t2"."name") > %s)) AS "t3" ON ("t3"."manifest_id" = "t1"."id") WHERE ("t1"."repository_id" = %s)', [False, None, 1691476694635, 4, 0, 4]) gunicorn-web stdout | 2023-08-08 06:38:14,638 [197] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,638 [197] [DEBUG] [peewee] ('UPDATE "repositorysize" SET "size_bytes" = %s WHERE ("repositorysize"."repository_id" = %s)', [0, 4]) gunicorn-web stdout | 2023-08-08 06:38:14,642 [197] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:14,643 [197] [DEBUG] [app] Ending request: urn:request:654c8e92-9054-4b47-bea1-8f242b943309 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:654c8e92-9054-4b47-bea1-8f242b943309', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=false"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'user1_build_test', 'popularity': 'true', 'public': 'false', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:14,644 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=false"a=true&state=true HTTP/2.0" 200 280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.050 118 0.050) gunicorn-web stdout | 2023-08-08 06:38:14,645 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:14 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=false"a=true&state=true HTTP/1.0" 200 280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" notificationworker stdout | 2023-08-08 06:38:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:38:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:25.371682+00:00 (in 9.999536 seconds) notificationworker stdout | 2023-08-08 06:38:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:25 UTC)" (scheduled at 2023-08-08 06:38:15.371682+00:00) notificationworker stdout | 2023-08-08 06:38:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:38:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 15, 372587), True, datetime.datetime(2023, 8, 8, 6, 38, 15, 372587), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:38:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:38:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:38:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:25 UTC)" executed successfully builder stdout | 2023-08-08 06:38:16,215 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:38:16,224 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:38:16,224 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:38:16,229 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:18,766 [197] [DEBUG] [app] Starting request: urn:request:1dab2963-67fd-4477-a520-71ee72509efa (/api/v1/organization/user1_build_test) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:18,767 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:18,767 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:18,767 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:18,774 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:18,775 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:18,775 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:18,775 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:18,776 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:18,777 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:18,777 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:18,777 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:18,778 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:18,781 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:18,781 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:18,781 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:18,781 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t2" ON ("t1"."role_id" = "t2"."id") WHERE ("t1"."organization_id" = %s)', [4]) gunicorn-web stdout | 2023-08-08 06:38:18,782 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2023-08-08 06:38:18,784 [197] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "repositorypermission" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [2]) gunicorn-web stdout | 2023-08-08 06:38:18,785 [197] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "teammember" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [2]) gunicorn-web stdout | 2023-08-08 06:38:18,786 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:18,786 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:18,786 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:18,787 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:38:18,788 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:18,790 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691476698789, 0, 4, None]) gunicorn-web stdout | 2023-08-08 06:38:18,793 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:18,795 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:38:18,797 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:18,799 [197] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691476698798, 0, 4]) gunicorn-web stdout | 2023-08-08 06:38:18,802 [197] [DEBUG] [app] Ending request: urn:request:1dab2963-67fd-4477-a520-71ee72509efa (/api/v1/organization/user1_build_test) {'endpoint': 'api.organization', 'request_id': 'urn:request:1dab2963-67fd-4477-a520-71ee72509efa', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/user1_build_test', 'path': '/api/v1/organization/user1_build_test', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:18,803 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:18 +0000] "GET /api/v1/organization/user1_build_test HTTP/2.0" 200 782 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.038 488 0.037) gunicorn-web stdout | 2023-08-08 06:38:18,803 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:18 +0000] "GET /api/v1/organization/user1_build_test HTTP/1.0" 200 782 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:19,356 [199] [DEBUG] [app] Starting request: urn:request:27302b43-a103-4c94-a72f-8004eaa5dc0f (/api/v1/organization/user1_build_test/quota) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:19,356 [200] [DEBUG] [app] Starting request: urn:request:6c7b7bf9-2a7b-4d66-b7ec-8baee16487f6 (/api/v1/organization/user1_build_test/proxycache) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:19,356 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,356 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,356 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,356 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,357 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,357 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,365 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:19,365 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:19,365 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:19,365 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,365 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:19,365 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,365 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,365 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,365 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,365 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,365 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,366 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,366 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,366 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,367 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:19,368 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:19,369 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,369 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,369 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,370 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,371 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,371 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,371 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,371 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:38:19,372 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', True, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,373 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."quota_id", "t1"."quota_type_id", "t1"."percent_of_limit" FROM "quotalimits" AS "t1" INNER JOIN "quotatype" AS "t2" ON ("t1"."quota_type_id" = "t2"."id") WHERE ("t1"."quota_id" = %s)', [None]) gunicorn-web stdout | 2023-08-08 06:38:19,374 [199] [DEBUG] [app] Ending request: urn:request:27302b43-a103-4c94-a72f-8004eaa5dc0f (/api/v1/organization/user1_build_test/quota) {'endpoint': 'api.organizationquotalist', 'request_id': 'urn:request:27302b43-a103-4c94-a72f-8004eaa5dc0f', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/user1_build_test/quota', 'path': '/api/v1/organization/user1_build_test/quota', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:19,374 [200] [DEBUG] [app] Ending request: urn:request:6c7b7bf9-2a7b-4d66-b7ec-8baee16487f6 (/api/v1/organization/user1_build_test/proxycache) {'endpoint': 'api.organizationproxycacheconfig', 'request_id': 'urn:request:6c7b7bf9-2a7b-4d66-b7ec-8baee16487f6', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/user1_build_test/proxycache', 'path': '/api/v1/organization/user1_build_test/proxycache', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:19,375 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:19,375 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:19 +0000] "GET /api/v1/organization/user1_build_test/quota HTTP/1.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:19 +0000] "GET /api/v1/organization/user1_build_test/quota HTTP/2.0" 200 113 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 408 0.021) gunicorn-web stdout | 2023-08-08 06:38:19,375 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:19 +0000] "GET /api/v1/organization/user1_build_test/proxycache HTTP/2.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.021 70 0.021) gunicorn-web stdout | 2023-08-08 06:38:19,376 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:19 +0000] "GET /api/v1/organization/user1_build_test/proxycache HTTP/1.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gcworker stdout | 2023-08-08 06:38:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:38:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:29.417660+00:00 (in 9.999495 seconds) gcworker stdout | 2023-08-08 06:38:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:29 UTC)" (scheduled at 2023-08-08 06:38:19.417660+00:00) gcworker stdout | 2023-08-08 06:38:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267099418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:38:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:38:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:38:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:29 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:19 +0000] "GET /static/img/creditcards/credit.png HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 130 -) gunicorn-web stdout | 2023-08-08 06:38:19,501 [200] [DEBUG] [app] Starting request: urn:request:b462cbf6-278c-4a13-b54b-5a882b70cb41 (/api/v1/organization/user1_build_test/prototypes) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:19,501 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,501 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,502 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,508 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:19,508 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:19,508 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,509 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,509 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,509 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,509 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,510 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:19,513 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,513 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,513 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,514 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,515 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" LEFT OUTER JOIN "user" AS "t2" ON ("t2"."id" = "t1"."activating_user_id") LEFT OUTER JOIN "user" AS "t3" ON ("t3"."id" = "t1"."delegate_user_id") LEFT OUTER JOIN "team" AS "t4" ON ("t4"."id" = "t1"."delegate_team_id") LEFT OUTER JOIN "role" AS "t5" ON ("t5"."id" = "t1"."role_id") WHERE ("t1"."org_id" = %s)', [4]) gunicorn-web stdout | 2023-08-08 06:38:19,518 [200] [DEBUG] [app] Ending request: urn:request:b462cbf6-278c-4a13-b54b-5a882b70cb41 (/api/v1/organization/user1_build_test/prototypes) {'endpoint': 'api.permissionprototypelist', 'request_id': 'urn:request:b462cbf6-278c-4a13-b54b-5a882b70cb41', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/organization/user1_build_test/prototypes', 'path': '/api/v1/organization/user1_build_test/prototypes', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:19,518 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:19 +0000] "GET /api/v1/organization/user1_build_test/prototypes HTTP/2.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.020 70 0.019) gunicorn-web stdout | 2023-08-08 06:38:19,519 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:19 +0000] "GET /api/v1/organization/user1_build_test/prototypes HTTP/1.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:19,522 [200] [DEBUG] [app] Starting request: urn:request:c83d517b-b2c0-4df1-b1ff-dabee5174337 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:19,522 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,522 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,523 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,529 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:19,529 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:19,529 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,530 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:19,530 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,530 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,530 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:19,531 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,532 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2023-08-08 06:38:19,535 [200] [DEBUG] [peewee] ('(((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t1"."visibility_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t6"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'user1_build_test', 1, 3, 1, 'user1_build_test', 3, 3, 1, 'user1_build_test', 3, 3, 1, 'user1_build_test', 1, 3, 101]) gunicorn-web stdout | 2023-08-08 06:38:19,541 [200] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Max("t1"."lifetime_start_ms") FROM "tag" AS "t1" WHERE ((("t1"."repository_id" IN (%s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) GROUP BY "t1"."repository_id"', [4, None, 1691476699541, False]) gunicorn-web stdout | 2023-08-08 06:38:19,543 [200] [DEBUG] [peewee] ('SELECT "t1"."repository_id", Sum("t1"."count") FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" IN (%s)) AND ("t1"."date" >= %s)) GROUP BY "t1"."repository_id"', [4, datetime.date(2023, 8, 1)]) gunicorn-web stdout | 2023-08-08 06:38:19,545 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [3, 1, 3]) gunicorn-web stdout | 2023-08-08 06:38:19,548 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,549 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1_build_test']) gunicorn-web stdout | 2023-08-08 06:38:19,551 [200] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,552 [200] [DEBUG] [peewee] ('SELECT Sum("t1"."layers_compressed_size") AS "size_bytes" FROM "manifest" AS "t1" INNER JOIN (SELECT "t2"."manifest_id" FROM "tag" AS "t2" WHERE ((("t2"."hidden" = %s) AND (("t2"."lifetime_end_ms" IS %s) OR ("t2"."lifetime_end_ms" > %s))) AND ("t2"."repository_id" = %s)) GROUP BY "t2"."manifest_id" HAVING (Count("t2"."name") > %s)) AS "t3" ON ("t3"."manifest_id" = "t1"."id") WHERE ("t1"."repository_id" = %s)', [False, None, 1691476699552, 4, 0, 4]) gunicorn-web stdout | 2023-08-08 06:38:19,554 [200] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,555 [200] [DEBUG] [peewee] ('UPDATE "repositorysize" SET "size_bytes" = %s WHERE ("repositorysize"."repository_id" = %s)', [0, 4]) gunicorn-web stdout | 2023-08-08 06:38:19,558 [200] [DEBUG] [peewee] ('SELECT "t1"."name", "t1"."id", "t2"."size_bytes" FROM "repositorysize" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") WHERE ("t2"."repository_id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:19,560 [200] [DEBUG] [app] Ending request: urn:request:c83d517b-b2c0-4df1-b1ff-dabee5174337 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:c83d517b-b2c0-4df1-b1ff-dabee5174337', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=true"a=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'user1_build_test', 'popularity': 'true', 'public': 'true', 'quota': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:19,561 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:19 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=true"a=true HTTP/2.0" 200 280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.040 110 0.040) gunicorn-web stdout | 2023-08-08 06:38:19,561 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:19 +0000] "GET /api/v1/repository?last_modified=true&namespace=user1_build_test&popularity=true&public=true"a=true HTTP/1.0" 200 280 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/organization/user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityworker stdout | 2023-08-08 06:38:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:38:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:26.890741+00:00 (in 7.202888 seconds) securityworker stdout | 2023-08-08 06:38:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:49 UTC)" (scheduled at 2023-08-08 06:38:19.687363+00:00) securityworker stdout | 2023-08-08 06:38:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:38:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:38:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:38:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:38:19,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:38:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:38:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 33, 19, 691766), 1, 2]) securityworker stdout | 2023-08-08 06:38:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 33, 19, 691766), 1, 2]) securityworker stdout | 2023-08-08 06:38:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:38:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:49 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:38:19,753 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:38:20,736 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:38:21,926 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:38:22,180 [197] [DEBUG] [app] Starting request: urn:request:6159d925-72af-4639-bc5e-31b64aa4855f (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:38:22,180 [199] [DEBUG] [app] Starting request: urn:request:cd894573-d2f9-4338-9282-1b7855270d96 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:38:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:22,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:38:22,185 [207] [DEBUG] [app] Starting request: urn:request:f16d2046-6793-47a8-b9e4-ad1035b1e377 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:38:22,185 [207] [DEBUG] [app] Ending request: urn:request:f16d2046-6793-47a8-b9e4-ad1035b1e377 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f16d2046-6793-47a8-b9e4-ad1035b1e377', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:38:22,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:38:22,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:38:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:38:22,186 [210] [DEBUG] [app] Starting request: urn:request:01bb3084-4c2c-4982-852c-5270c78f3b12 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:38:22,187 [210] [DEBUG] [app] Ending request: urn:request:01bb3084-4c2c-4982-852c-5270c78f3b12 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:01bb3084-4c2c-4982-852c-5270c78f3b12', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:38:22,187 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:38:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:22,188 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:22,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:22,191 [200] [DEBUG] [app] Starting request: urn:request:80f87054-6d66-4a74-8003-6dc0217f66ad (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:38:22,191 [200] [DEBUG] [app] Ending request: urn:request:80f87054-6d66-4a74-8003-6dc0217f66ad (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:80f87054-6d66-4a74-8003-6dc0217f66ad', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:38:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:38:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:22,193 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:22,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:38:22,193 [200] [DEBUG] [app] Starting request: urn:request:fb8a970b-b765-4519-b0c6-8254dd879a2c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:38:22,194 [200] [DEBUG] [app] Ending request: urn:request:fb8a970b-b765-4519-b0c6-8254dd879a2c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fb8a970b-b765-4519-b0c6-8254dd879a2c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:38:22,194 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:38:22,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:22,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:38:22,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:38:22,200 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:38:22,203 [199] [DEBUG] [app] Ending request: urn:request:cd894573-d2f9-4338-9282-1b7855270d96 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cd894573-d2f9-4338-9282-1b7855270d96', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:38:22,203 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:38:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:38:22,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:38:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:38:22,205 [197] [DEBUG] [app] Ending request: urn:request:6159d925-72af-4639-bc5e-31b64aa4855f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6159d925-72af-4639-bc5e-31b64aa4855f', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:38:22,206 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:38:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) gunicorn-web stdout | 2023-08-08 06:38:22,206 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:38:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:38:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:38:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:28.874906+00:00 (in 6.580641 seconds) chunkcleanupworker stdout | 2023-08-08 06:38:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:32 UTC)" (scheduled at 2023-08-08 06:38:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:38:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:38:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 22, 294779), True, datetime.datetime(2023, 8, 8, 6, 38, 22, 294779), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:38:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:38:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:38:22,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:32 UTC)" executed successfully nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:22 +0000] "GET /static/fonts/glyphicons-halflings-regular.woff2 HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/static/ldn/bootstrap.min.css-13f668494557.css" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 586 -) storagereplication stdout | 2023-08-08 06:38:22,420 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gcworker stdout | 2023-08-08 06:38:25,056 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:38:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:38:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:35.371682+00:00 (in 9.999544 seconds) notificationworker stdout | 2023-08-08 06:38:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:35 UTC)" (scheduled at 2023-08-08 06:38:25.371682+00:00) notificationworker stdout | 2023-08-08 06:38:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:38:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 25, 372576), True, datetime.datetime(2023, 8, 8, 6, 38, 25, 372576), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:38:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:38:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:38:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:38:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:38:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:33.514094+00:00 (in 7.999013 seconds) exportactionlogsworker stdout | 2023-08-08 06:38:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:25 UTC)" (scheduled at 2023-08-08 06:38:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:38:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:38:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:25 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:38:25,543 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:38:25,547 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:38:25,550 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:38:26,230 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:38:26,239 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:38:26,239 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:38:26,243 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:38:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:38:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:49.687363+00:00 (in 22.796117 seconds) securityworker stdout | 2023-08-08 06:38:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:56 UTC)" (scheduled at 2023-08-08 06:38:26.890741+00:00) securityworker stdout | 2023-08-08 06:38:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:38:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:38:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:38:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:38:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:38:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:38:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:38:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:38:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:38:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:38:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 33, 26, 896385), 1, 2]) securityworker stdout | 2023-08-08 06:38:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 33, 26, 896385), 1, 2]) securityworker stdout | 2023-08-08 06:38:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:38:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:38:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:38:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:38:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:38:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:38:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:38:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:38:56 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:38:26,939 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:38:27,320 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:38:27,323 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:38:27,938 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:38:28,231 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:38:28,349 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:38:28,361 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:38:28,364 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} teamsyncworker stdout | 2023-08-08 06:38:28,367 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} repositoryactioncounter stdout | 2023-08-08 06:38:28,848 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} chunkcleanupworker stdout | 2023-08-08 06:38:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:38:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:32.293757+00:00 (in 3.418359 seconds) chunkcleanupworker stdout | 2023-08-08 06:38:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:28 UTC)" (scheduled at 2023-08-08 06:38:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:38:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:38:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:28 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:38:29,041 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:38:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:38:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:39.417660+00:00 (in 9.999521 seconds) gcworker stdout | 2023-08-08 06:38:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:39 UTC)" (scheduled at 2023-08-08 06:38:29.417660+00:00) gcworker stdout | 2023-08-08 06:38:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267109419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:38:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:38:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:38:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:39 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:29,848 [197] [DEBUG] [app] Starting request: urn:request:282f3f96-eb66-43c3-bee9-293f68ed6ab9 (/api/v1/repository) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:29,848 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:29,848 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:29,849 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,856 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:29,856 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:29,856 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:29,857 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:29,857 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:29,857 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:29,857 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:29,858 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:29,861 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:29,861 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:29,861 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:29,862 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,865 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,866 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,868 [197] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [4, 'gitlab', 2, '', '67fb97de-c330-4f8c-a9eb-00e0f5bd8ecb', 1, False, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,871 [197] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [5, 0, datetime.date(2023, 8, 7)]) gunicorn-web stdout | 2023-08-08 06:38:29,872 [197] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [5, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,873 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,874 [197] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [3, 5, 1]) gunicorn-web stdout | 2023-08-08 06:38:29,877 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_user_id" IS %s))', [4, 3, None, None]) gunicorn-web stdout | 2023-08-08 06:38:29,879 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_team_id" IS %s))', [4, 3, None, None]) gunicorn-web stdout | 2023-08-08 06:38:29,881 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,882 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:29,884 [197] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [14, 4, 3, 5, datetime.datetime(2023, 8, 8, 6, 38, 29, 882280), '10.130.2.1', '{"repo": "gitlab", "namespace": "user1_build_test"}']) gunicorn-web stdout | 2023-08-08 06:38:29,889 [197] [DEBUG] [app] Ending request: urn:request:282f3f96-eb66-43c3-bee9-293f68ed6ab9 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:282f3f96-eb66-43c3-bee9-293f68ed6ab9', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository', 'path': '/api/v1/repository', 'parameters': {}, 'json_body': {'namespace': 'user1_build_test', 'repository': 'gitlab', 'visibility': 'private', 'description': '', 'repo_kind': 'image'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:29,889 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:29,890 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:29 +0000] "POST /api/v1/repository HTTP/1.0" 201 69 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:29 +0000] "POST /api/v1/repository HTTP/2.0" 201 69 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/new/?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.043 1051 0.043) gunicorn-web stdout | 2023-08-08 06:38:30,256 [200] [DEBUG] [app] Starting request: urn:request:dd106823-2fc8-4cd2-ac01-2b8fe5649270 (/api/v1/repository/user1_build_test/gitlab/tag/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:30,256 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,256 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,257 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:30,258 [197] [DEBUG] [app] Starting request: urn:request:89b3468a-13a5-4f07-86e4-8c3d528deb6e (/api/v1/repository/user1_build_test/gitlab) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:30,258 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,258 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,259 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:30,264 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:30,264 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:30,264 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,265 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:30,265 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,265 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:30,265 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:30,265 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:30,265 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:30,266 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:30,266 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,266 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:30,266 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,266 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:30,266 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:30,266 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:30,267 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:30,268 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:30,273 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:30,274 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:30,275 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:30,275 [197] [DEBUG] [endpoints.api.repository] Get repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:30,276 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:30,278 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:30,278 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [5, 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:30,279 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [5, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:38:30,279 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [5, None, 1691476710279, False, 101, 0]) gunicorn-web stdout | 2023-08-08 06:38:30,281 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,281 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:30,282 [197] [DEBUG] [app] Ending request: urn:request:89b3468a-13a5-4f07-86e4-8c3d528deb6e (/api/v1/repository/user1_build_test/gitlab) {'endpoint': 'api.repository', 'request_id': 'urn:request:89b3468a-13a5-4f07-86e4-8c3d528deb6e', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab?includeStats=true&includeTags=false', 'path': '/api/v1/repository/user1_build_test/gitlab', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:30,282 [200] [DEBUG] [app] Ending request: urn:request:dd106823-2fc8-4cd2-ac01-2b8fe5649270 (/api/v1/repository/user1_build_test/gitlab/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:dd106823-2fc8-4cd2-ac01-2b8fe5649270', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/user1_build_test/gitlab/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:30,282 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:30,283 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:30,283 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:30 +0000] "GET /api/v1/repository/user1_build_test/gitlab?includeStats=true&includeTags=false HTTP/1.0" 200 3632 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:30,283 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:30 +0000] "GET /api/v1/repository/user1_build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:30 +0000] "GET /api/v1/repository/user1_build_test/gitlab?includeStats=true&includeTags=false HTTP/2.0" 200 3632 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 92 0.026) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:30 +0000] "GET /api/v1/repository/user1_build_test/gitlab/tag/?limit=100&page=1&onlyActiveTags=true HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 545 0.029) buildlogsarchiver stdout | 2023-08-08 06:38:30,432 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:38:30,602 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} gunicorn-web stdout | 2023-08-08 06:38:31,023 [197] [DEBUG] [app] Starting request: urn:request:fdee2ffb-4716-47f9-b969-719d98d5e18c (/repository/user1_build_test/gitlab/status) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:31,024 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,032 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,034 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 'public', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,037 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['gitlab', 'user1_build_test', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,043 [197] [DEBUG] [app] Ending request: urn:request:fdee2ffb-4716-47f9-b969-719d98d5e18c (/repository/user1_build_test/gitlab/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:fdee2ffb-4716-47f9-b969-719d98d5e18c', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/status?token=67fb97de-c330-4f8c-a9eb-00e0f5bd8ecb', 'path': '/repository/user1_build_test/gitlab/status', 'parameters': {'token': '67fb97de-c330-4f8c-a9eb-00e0f5bd8ecb'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:31,043 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:31,044 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:31 +0000] "GET /repository/user1_build_test/gitlab/status?token=67fb97de-c330-4f8c-a9eb-00e0f5bd8ecb HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:31 +0000] "GET /repository/user1_build_test/gitlab/status?token=67fb97de-c330-4f8c-a9eb-00e0f5bd8ecb HTTP/2.0" 200 387 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.023 469 0.022) gunicorn-web stdout | 2023-08-08 06:38:31,166 [199] [DEBUG] [app] Starting request: urn:request:a57fd058-f00b-4b3c-a57f-c4b2a593fe18 (/api/v1/repository/user1_build_test/gitlab/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:31,166 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,166 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,167 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,174 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:31,174 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:31,174 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,174 [199] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:31,174 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,174 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,174 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,174 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,176 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:31,183 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,184 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,188 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:31,192 [199] [DEBUG] [app] Ending request: urn:request:a57fd058-f00b-4b3c-a57f-c4b2a593fe18 (/api/v1/repository/user1_build_test/gitlab/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:a57fd058-f00b-4b3c-a57f-c4b2a593fe18', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/build/?limit=3', 'path': '/api/v1/repository/user1_build_test/gitlab/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:31,193 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:31 +0000] "GET /api/v1/repository/user1_build_test/gitlab/build/?limit=3 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.028 77 0.029) gunicorn-web stdout | 2023-08-08 06:38:31,193 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:31 +0000] "GET /api/v1/repository/user1_build_test/gitlab/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?namespace=user1_build_test" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityworker stdout | 2023-08-08 06:38:31,586 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} gunicorn-web stdout | 2023-08-08 06:38:31,838 [197] [DEBUG] [app] Starting request: urn:request:40aaaaca-17c1-4761-8c5b-6a295d8b6617 (/api/v1/repository/user1_build_test/gitlab/trigger/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:31,839 [200] [DEBUG] [app] Starting request: urn:request:63e62be6-5f1c-4a71-8174-46c1a34ebd45 (/api/v1/repository/user1_build_test/gitlab/build/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:31,839 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,839 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,839 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,839 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,839 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,840 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,847 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:31,847 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:31,847 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:31,847 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,847 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:31,847 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,847 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:31,847 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,847 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:31,847 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,847 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,847 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:31,847 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,847 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,847 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,847 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,849 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:31,850 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:31,856 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,856 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:31,857 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,857 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:31,860 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") WHERE (("t4"."username" = %s) AND ("t3"."name" = %s))', ['user1_build_test', 'gitlab']) gunicorn-web stdout | 2023-08-08 06:38:31,861 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['gitlab', 'user1_build_test', 10]) gunicorn-web stdout | 2023-08-08 06:38:31,862 [197] [DEBUG] [app] Ending request: urn:request:40aaaaca-17c1-4761-8c5b-6a295d8b6617 (/api/v1/repository/user1_build_test/gitlab/trigger/) {'endpoint': 'api.buildtriggerlist', 'request_id': 'urn:request:40aaaaca-17c1-4761-8c5b-6a295d8b6617', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:31,863 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:31 +0000] "GET /api/v1/repository/user1_build_test/gitlab/trigger/ HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 510 0.026) gunicorn-web stdout | 2023-08-08 06:38:31,863 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:31 +0000] "GET /api/v1/repository/user1_build_test/gitlab/trigger/ HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:31,865 [200] [DEBUG] [app] Ending request: urn:request:63e62be6-5f1c-4a71-8174-46c1a34ebd45 (/api/v1/repository/user1_build_test/gitlab/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:63e62be6-5f1c-4a71-8174-46c1a34ebd45', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/build/?limit=10', 'path': '/api/v1/repository/user1_build_test/gitlab/build/', 'parameters': {'limit': '10'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:31,866 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:31 +0000] "GET /api/v1/repository/user1_build_test/gitlab/build/?limit=10 HTTP/2.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.029 77 0.029) gunicorn-web stdout | 2023-08-08 06:38:31,866 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:31 +0000] "GET /api/v1/repository/user1_build_test/gitlab/build/?limit=10 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab?tab=builds" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" chunkcleanupworker stdout | 2023-08-08 06:38:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:38:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:42.293757+00:00 (in 9.999494 seconds) chunkcleanupworker stdout | 2023-08-08 06:38:32,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:42 UTC)" (scheduled at 2023-08-08 06:38:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:38:32,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:38:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 32, 295151), True, datetime.datetime(2023, 8, 8, 6, 38, 32, 295151), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:38:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:38:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:38:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:38:32,497 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:38:32,600 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:38:32,981 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} exportactionlogsworker stdout | 2023-08-08 06:38:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:38:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:47.398551+00:00 (in 13.884002 seconds) exportactionlogsworker stdout | 2023-08-08 06:38:33,514 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:43:33 UTC)" (scheduled at 2023-08-08 06:38:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:38:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 38, 33, 514960), 'exportactionlogs/%']) securityscanningnotificationworker stdout | 2023-08-08 06:38:33,518 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} exportactionlogsworker stdout | 2023-08-08 06:38:33,522 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 38, 33, 514960), True, datetime.datetime(2023, 8, 8, 6, 38, 33, 514960), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:38:33,524 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 38, 33, 514960), True, datetime.datetime(2023, 8, 8, 6, 38, 33, 514960), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 38, 33, 514960), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:38:33,526 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:38:33,526 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:43:33 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:38:33,825 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:38:33,909 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:38:33,913 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:38:33,916 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:38:33,919 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:38:33,923 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:38:33,925 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:38:33,931 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:38:33,935 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:38:33,940 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:38:34,303 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:38:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:38:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:45.371682+00:00 (in 9.999466 seconds) notificationworker stdout | 2023-08-08 06:38:35,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:45 UTC)" (scheduled at 2023-08-08 06:38:35.371682+00:00) notificationworker stdout | 2023-08-08 06:38:35,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:38:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 35, 373180), True, datetime.datetime(2023, 8, 8, 6, 38, 35, 373180), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:38:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:38:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:38:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:45 UTC)" executed successfully builder stdout | 2023-08-08 06:38:36,253 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:38:36,263 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:38:36,263 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:38:36,268 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:38:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:38:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:54.393627+00:00 (in 17.813187 seconds) namespacegcworker stdout | 2023-08-08 06:38:36,581 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:36 UTC)" (scheduled at 2023-08-08 06:38:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:38:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:38:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:36,680 [199] [DEBUG] [app] Starting request: urn:request:5c8d63b5-27b6-474b-acf5-2f6fd55b76b7 (/oauth2/gitlab/callback/trigger) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:36,680 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:36,680 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:36,681 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:36,688 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:36,688 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:36,688 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:36,688 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:36,688 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:36,688 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:36,690 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:36,697 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:36,700 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:36,917 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "POST /oauth/token?code=7f59e034de8e28de163628d2b7f84c6181e21933b756254decd828c830fc35fa&grant_type=authorization_code&redirect_uri=https%3A%2F%2Fquayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com%2Foauth2%2Fgitlab%2Fcallback%2Ftrigger&client_id=e6224cb14edfcc638a26b98e3708b3cd521e79e5600afa1dba37ee297b9fc38e&client_secret=ccbbdfc399411f5a1a684ef5280e5e27954593d237067c84aaa9db50c5aa2169 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:36,918 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:36,921 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:36,922 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "buildtriggerservice" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['gitlab', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:36,923 [199] [DEBUG] [peewee] ('INSERT INTO "repositorybuildtrigger" ("uuid", "service_id", "repository_id", "connected_user_id", "secure_auth_token", "fully_migrated", "config", "pull_robot_id", "enabled", "disabled_datetime", "successive_failure_count", "successive_internal_error_count") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repositorybuildtrigger"."id"', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 2, 5, 3, 'v0$$PCwBgEmfg29TDtk7U46vWznezSitslrPYv4Gsa0h1YjlCSpSFdOhuB1Y+K4QHgrNJo11X01OTqAFGzsQtKmATZOFIK3kW6wRQmtWXW8/61D9KLthR4OPH+bQ2F0n', False, '{}', None, True, datetime.datetime(2023, 8, 8, 6, 38, 36, 923247), 0, 0]) gunicorn-web stdout | 2023-08-08 06:38:36,930 [199] [DEBUG] [endpoints.gitlabtrigger] Redirecting to full url: /repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e gunicorn-web stdout | 2023-08-08 06:38:36,930 [199] [DEBUG] [app] Ending request: urn:request:5c8d63b5-27b6-474b-acf5-2f6fd55b76b7 (/oauth2/gitlab/callback/trigger) {'endpoint': 'gitlab.attach_gitlab_build_trigger', 'request_id': 'urn:request:5c8d63b5-27b6-474b-acf5-2f6fd55b76b7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/oauth2/gitlab/callback/trigger?code=7f59e034de8e28de163628d2b7f84c6181e21933b756254decd828c830fc35fa&state=repo%3Auser1_build_test%2Fgitlab', 'path': '/oauth2/gitlab/callback/trigger', 'parameters': {'code': '7f59e034de8e28de163628d2b7f84c6181e21933b756254decd828c830fc35fa', 'state': 'repo:user1_build_test/gitlab'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:36,931 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:36 +0000] "GET /oauth2/gitlab/callback/trigger?code=7f59e034de8e28de163628d2b7f84c6181e21933b756254decd828c830fc35fa&state=repo%3Auser1_build_test%2Fgitlab HTTP/2.0" 302 367 "https://gitlab.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.253 262 0.252) gunicorn-web stdout | 2023-08-08 06:38:36,932 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:36 +0000] "GET /oauth2/gitlab/callback/trigger?code=7f59e034de8e28de163628d2b7f84c6181e21933b756254decd828c830fc35fa&state=repo%3Auser1_build_test%2Fgitlab HTTP/1.0" 302 367 "https://gitlab.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:37,178 [199] [DEBUG] [app] Starting request: urn:request:bb83c92e-1f51-4b57-bfa1-b56afbe1a7c0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:38:37,179 [200] [DEBUG] [app] Starting request: urn:request:40c8ad72-1302-4f78-a636-7e49c83ef3c6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:38:37,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:37,182 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:38:37,183 [210] [DEBUG] [app] Starting request: urn:request:8a7f230b-c2c2-4913-9431-1ff77e5f042d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:38:37,184 [209] [DEBUG] [app] Starting request: urn:request:4c28d609-7b61-426a-83cb-efd0d8649a1c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:38:37,184 [210] [DEBUG] [app] Ending request: urn:request:8a7f230b-c2c2-4913-9431-1ff77e5f042d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8a7f230b-c2c2-4913-9431-1ff77e5f042d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:38:37,184 [209] [DEBUG] [app] Ending request: urn:request:4c28d609-7b61-426a-83cb-efd0d8649a1c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4c28d609-7b61-426a-83cb-efd0d8649a1c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:38:37,184 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:38:37,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:38:37,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:38:37,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:37,186 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:37,186 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:37,188 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:37,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:37,190 [200] [DEBUG] [app] Starting request: urn:request:3dfbb832-a680-4390-9223-7c21b33d9f6c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:38:37,190 [200] [DEBUG] [app] Ending request: urn:request:3dfbb832-a680-4390-9223-7c21b33d9f6c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3dfbb832-a680-4390-9223-7c21b33d9f6c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:38:37,190 [197] [DEBUG] [app] Starting request: urn:request:9d415e36-fbfb-409a-a982-a347430621ad (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:38:37,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:38:37,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:37,191 [197] [DEBUG] [app] Ending request: urn:request:9d415e36-fbfb-409a-a982-a347430621ad (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9d415e36-fbfb-409a-a982-a347430621ad', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:38:37,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:38:37,191 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:38:37,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:37,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:38:37,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:38:37,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:38:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:38:37,197 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:07.197412+00:00 (in 29.999572 seconds) buildlogsarchiver stdout | 2023-08-08 06:38:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:07 UTC)" (scheduled at 2023-08-08 06:38:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:38:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 38, 37, 198247), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:37,202 [200] [DEBUG] [app] Ending request: urn:request:40c8ad72-1302-4f78-a636-7e49c83ef3c6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:40c8ad72-1302-4f78-a636-7e49c83ef3c6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:38:37,202 [199] [DEBUG] [app] Ending request: urn:request:bb83c92e-1f51-4b57-bfa1-b56afbe1a7c0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bb83c92e-1f51-4b57-bfa1-b56afbe1a7c0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:38:37,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:37,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:38:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:38:37,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:38:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:38:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:38:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) buildlogsarchiver stdout | 2023-08-08 06:38:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:38:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:38:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:38:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:38:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:07 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:37,336 [198] [DEBUG] [app] Starting request: urn:request:f89bea7b-f27d-44b2-9918-0aa1731232f6 (/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:37,338 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:37,340 [198] [DEBUG] [app] Ending request: urn:request:f89bea7b-f27d-44b2-9918-0aa1731232f6 (/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e) {'endpoint': 'web.buildtrigger', 'request_id': 'urn:request:f89bea7b-f27d-44b2-9918-0aa1731232f6', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e', 'path': '/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:37,341 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:37 +0000] "GET /repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e HTTP/1.0" 200 85192 "https://gitlab.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:37 +0000] "GET /repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e HTTP/2.0" 200 13315 "https://gitlab.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.006 544 0.007) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/jquery.js-3c5c5966471e.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 675 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/bootstrap.min.js-0e773844b374.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 84 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/angular.min.js-f0f4228b82f7.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 83 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/angular-route.min.js-590b72b0eb88.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 86 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/angular-sanitize.min.js-ff589de0ec1b.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 88 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/angular-animate.min.js-636f7a51b2e1.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 88 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/angular-cookies.min.js-2455ea25d522.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 86 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/momentjs-66a52705d33f.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 78 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/bootstrap-datepicker.min.js-3e3d8e141d5e.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 91 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/bootstrap-datetimepicker.min.js-6612bb423b71.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 94 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/bootbox4.1.0underscorejs1.5.2restangular1.2.0d3js3.3.3-6d47ddfef9c5.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 111 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/font-awesome.css-a166e3c966c6.css HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 105 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/raven.min.js-ecd480647a39.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 81 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/bootstrap.min.css-13f668494557.css HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 85 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/cal-heatmap.min.js-8ffad728269a.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 85 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/cssfamilySourceSansPro300400700-30cd793a4b9b.css HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 93 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/angular-recaptcha.min.js-23859fca3602.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 95 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/bootstrap-datetimepicker.min.css-4b15a4ede750.css HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 94 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/cal-heatmap.css-904c3b9e8a60.css HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 82 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/ng-tags-input.min.js-fa3026ba8409.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 86 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/ng-tags-input.min.css-7782190f81cd.css HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 86 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/ldn/typeahead.bundle.min.js-716151ca4e5d.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 88 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/build/main-quay-frontend-ab93c42f539d59de0115.bundle.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 122 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/standalonelib/jquery.overscroll.min.js HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 85 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/img/RedHat.svg HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 94 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/img/network-tile.png HTTP/2.0" 200 6289 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 54 -) gunicorn-web stdout | 2023-08-08 06:38:38,887 [199] [DEBUG] [app] Starting request: urn:request:6aeb08d3-8cb9-48a2-9779-01ca2c944677 (/api/v1/user/) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:38,887 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,887 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,888 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:38,895 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:38,896 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:38,896 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,896 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,896 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,896 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,896 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,897 [199] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'user1']) gunicorn-web stdout | 2023-08-08 06:38:38,900 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,900 [199] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 3]) gunicorn-web stdout | 2023-08-08 06:38:38,902 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [3]) gunicorn-web stdout | 2023-08-08 06:38:38,903 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:38,905 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:38:38,906 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:38,908 [199] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t2"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1691476718907, 0, 3, None]) gunicorn-web stdout | 2023-08-08 06:38:38,911 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:38,913 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['user1']) gunicorn-web stdout | 2023-08-08 06:38:38,914 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['user1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:38,915 [197] [DEBUG] [app] Starting request: urn:request:607475d7-8b3c-4934-b46e-a64b99183dee (/api/v1/messages) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:38,915 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,915 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,916 [199] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."manifest_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."manifest_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."manifest_id" = "t1"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1691476718915, 0, 3]) gunicorn-web stdout | 2023-08-08 06:38:38,916 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:38,917 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,917 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,919 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:38,921 [199] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,921 [199] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,921 [199] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,921 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,921 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,922 [199] [DEBUG] [app] Ending request: urn:request:6aeb08d3-8cb9-48a2-9779-01ca2c944677 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:6aeb08d3-8cb9-48a2-9779-01ca2c944677', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:38,922 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:38,923 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:38 +0000] "GET /api/v1/user/ HTTP/1.0" 200 918 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /api/v1/user/ HTTP/2.0" 200 918 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.037 172 0.037) gunicorn-web stdout | 2023-08-08 06:38:38,923 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:38,923 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:38,923 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,924 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2023-08-08 06:38:38,926 [197] [DEBUG] [app] Ending request: urn:request:607475d7-8b3c-4934-b46e-a64b99183dee (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:607475d7-8b3c-4934-b46e-a64b99183dee', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:38,927 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /api/v1/messages HTTP/2.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 157 0.013) gunicorn-web stdout | 2023-08-08 06:38:38,927 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:38 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:38 +0000] "GET /static/img/RH_Logo_Quay_Black_UX-horizontal.svg HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 88 -) gunicorn-web stdout | 2023-08-08 06:38:38,989 [199] [DEBUG] [app] Starting request: urn:request:23007cf3-49b9-4f5e-b14c-654b392130b6 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:38,989 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,989 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,989 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:38,989 [197] [DEBUG] [app] Starting request: urn:request:67e22a6b-731a-4a0d-889b-b7694147bb04 (/api/v1/repository/user1_build_test/gitlab) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:38,990 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,990 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,991 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:38,996 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:38,996 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:38,996 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,997 [199] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:38,997 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,997 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,997 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,997 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,997 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:38,997 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:38,997 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,998 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:38,998 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:38,998 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,998 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,998 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:38,999 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:39,000 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:39,005 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,006 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,006 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,006 [197] [DEBUG] [endpoints.api.repository] Get repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:39,007 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,008 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [5, 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,009 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,010 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [5, datetime.date(2023, 5, 8)]) gunicorn-web stdout | 2023-08-08 06:38:39,011 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,011 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,012 [199] [DEBUG] [app] Ending request: urn:request:23007cf3-49b9-4f5e-b14c-654b392130b6 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e) {'endpoint': 'api.buildtrigger', 'request_id': 'urn:request:23007cf3-49b9-4f5e-b14c-654b392130b6', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:39,012 [197] [DEBUG] [app] Ending request: urn:request:67e22a6b-731a-4a0d-889b-b7694147bb04 (/api/v1/repository/user1_build_test/gitlab) {'endpoint': 'api.repository', 'request_id': 'urn:request:67e22a6b-731a-4a0d-889b-b7694147bb04', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab?includeTags=false', 'path': '/api/v1/repository/user1_build_test/gitlab', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:39,012 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:39,013 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:39 +0000] "GET /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e HTTP/1.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:39,013 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:39 +0000] "GET /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e HTTP/2.0" 200 210 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.026 98 0.026) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:39 +0000] "GET /api/v1/repository/user1_build_test/gitlab?includeTags=false HTTP/2.0" 200 345 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.025 79 0.025) gunicorn-web stdout | 2023-08-08 06:38:39,013 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:39 +0000] "GET /api/v1/repository/user1_build_test/gitlab?includeTags=false HTTP/1.0" 200 345 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:39 +0000] "GET /static/fonts/fontawesome-webfont.woff2?v=4.7.0 HTTP/2.0" 304 0 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/static/ldn/font-awesome.css-a166e3c966c6.css" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 244 -) nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:39 +0000] "GET /static/img/quay_favicon.png HTTP/2.0" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.000 615 -) gcworker stdout | 2023-08-08 06:38:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:38:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:49.417660+00:00 (in 9.999421 seconds) gcworker stdout | 2023-08-08 06:38:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:49 UTC)" (scheduled at 2023-08-08 06:38:39.417660+00:00) gcworker stdout | 2023-08-08 06:38:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267119418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:38:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:38:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:38:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:49 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:39,465 [197] [DEBUG] [app] Starting request: urn:request:a7bf2967-f512-4766-bf2f-9c1c92170674 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:39,465 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,465 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,466 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,467 [198] [DEBUG] [app] Starting request: urn:request:633b6d3a-1a9f-4b8b-8873-c8fa78a1b208 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:39,468 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,468 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,468 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,472 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:39,472 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:39,472 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,472 [197] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:38:39,472 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,472 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,472 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,472 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,474 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:38:39,474 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:39,474 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:39,474 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,474 [198] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:38:39,474 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,474 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,474 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,475 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,475 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 50, 39, 475150), datetime.datetime(2023, 8, 8, 6, 38, 39, 475211)]) gunicorn-web stdout | 2023-08-08 06:38:39,477 [198] [DEBUG] [app] Ending request: urn:request:633b6d3a-1a9f-4b8b-8873-c8fa78a1b208 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:633b6d3a-1a9f-4b8b-8873-c8fa78a1b208', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:39,478 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:39 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.012 62 0.011) gunicorn-web stdout | 2023-08-08 06:38:39,478 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:39 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:39,478 [197] [DEBUG] [app] Ending request: urn:request:a7bf2967-f512-4766-bf2f-9c1c92170674 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:a7bf2967-f512-4766-bf2f-9c1c92170674', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:39,479 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:39 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.016 54 0.016) gunicorn-web stdout | 2023-08-08 06:38:39,479 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:39 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:39,535 [200] [DEBUG] [app] Starting request: urn:request:58abb63a-7604-4456-9c88-17f9f7abaf73 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/namespaces) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:39,535 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,535 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,536 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,542 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:39,542 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:39,542 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,543 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:39,543 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,543 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,543 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,543 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,545 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:39,551 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:39,552 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,555 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,558 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:39,560 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:39,562 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:39,828 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:40,072 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/namespaces?page=1&per_page=20 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:40,074 [200] [DEBUG] [app] Ending request: urn:request:58abb63a-7604-4456-9c88-17f9f7abaf73 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/namespaces) {'endpoint': 'api.buildtriggersourcenamespaces', 'request_id': 'urn:request:58abb63a-7604-4456-9c88-17f9f7abaf73', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/namespaces', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/namespaces', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:40,075 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:40 +0000] "GET /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/namespaces HTTP/2.0" 200 365 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.542 105 0.542) gunicorn-web stdout | 2023-08-08 06:38:40,075 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:40 +0000] "GET /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/namespaces HTTP/1.0" 200 365 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:41,979 [197] [DEBUG] [app] Starting request: urn:request:62999636-453b-43d5-a64f-5c26ae8b23b0 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/sources) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:41,979 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:41,979 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:41,980 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:41,986 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:41,987 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:41,987 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:41,987 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:41,987 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:41,987 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:41,987 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:41,987 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:41,989 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:41,995 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:41,996 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:41,999 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:42,002 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:42,005 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:42,006 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:42,008 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:42,191 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None chunkcleanupworker stdout | 2023-08-08 06:38:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:38:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:52.293757+00:00 (in 9.999550 seconds) chunkcleanupworker stdout | 2023-08-08 06:38:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:52 UTC)" (scheduled at 2023-08-08 06:38:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:38:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:38:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 42, 294669), True, datetime.datetime(2023, 8, 8, 6, 38, 42, 294669), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:38:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:38:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:38:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:52 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:42,625 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/namespaces/52807017 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:43,026 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/groups/52807017/projects?page=1&per_page=20 HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:43,029 [197] [DEBUG] [app] Ending request: urn:request:62999636-453b-43d5-a64f-5c26ae8b23b0 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/sources) {'endpoint': 'api.buildtriggersources', 'request_id': 'urn:request:62999636-453b-43d5-a64f-5c26ae8b23b0', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/sources', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/sources', 'parameters': {}, 'json_body': {'namespace': '52807017'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:43,031 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:43 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/sources HTTP/2.0" 200 503 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.054 575 1.054) gunicorn-web stdout | 2023-08-08 06:38:43,031 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:43 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/sources HTTP/1.0" 200 503 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:38:44,711 [197] [DEBUG] [app] Starting request: urn:request:7af29b8c-dc25-4765-8c2b-a2140ee7bf41 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/subdir) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:44,711 [200] [DEBUG] [app] Starting request: urn:request:5ce3bf8b-114d-4e4b-8713-3b3167445baa (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/fields/refs) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:44,712 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,712 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,712 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,712 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,712 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,712 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,720 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:44,720 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:44,720 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:44,720 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:44,720 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,720 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,720 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:44,720 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:44,720 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,720 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,720 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:44,720 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:44,721 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:44,721 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:44,721 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:44,721 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:44,723 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:44,723 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:44,729 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:44,730 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:44,730 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,731 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,733 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,733 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,736 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,736 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,738 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,739 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:44,740 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:44,740 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:44,743 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:44,939 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:44,946 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:45,192 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None notificationworker stdout | 2023-08-08 06:38:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:38:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:49.178248+00:00 (in 3.806028 seconds) notificationworker stdout | 2023-08-08 06:38:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:55 UTC)" (scheduled at 2023-08-08 06:38:45.371682+00:00) notificationworker stdout | 2023-08-08 06:38:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:38:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 45, 372643), True, datetime.datetime(2023, 8, 8, 6, 38, 45, 372643), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:38:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:38:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:38:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:55 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:45,419 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:45,423 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:45,429 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:45,600 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:45,618 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:45,804 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/tree?recursive=False&ref=main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:45,805 [197] [DEBUG] [app] Ending request: urn:request:7af29b8c-dc25-4765-8c2b-a2140ee7bf41 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/subdir) {'endpoint': 'api.buildtriggersubdirs', 'request_id': 'urn:request:7af29b8c-dc25-4765-8c2b-a2140ee7bf41', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/subdir', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/subdir', 'parameters': {}, 'json_body': {'build_source': 'h4051/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:45,806 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:45 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/subdir HTTP/2.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.096 146 1.095) gunicorn-web stdout | 2023-08-08 06:38:45,806 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:45 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/subdir HTTP/1.0" 200 97 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" builder stdout | 2023-08-08 06:38:46,277 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:38:46,287 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:38:46,287 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:38:46,291 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:46,497 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:46,711 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/tags HTTP/1.1" 200 2 gunicorn-web stdout | 2023-08-08 06:38:46,712 [200] [DEBUG] [app] Ending request: urn:request:5ce3bf8b-114d-4e4b-8713-3b3167445baa (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/fields/refs) {'endpoint': 'api.buildtriggerfieldvalues', 'request_id': 'urn:request:5ce3bf8b-114d-4e4b-8713-3b3167445baa', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/fields/refs', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/fields/refs', 'parameters': {}, 'json_body': {'build_source': 'h4051/quay_build_test'}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:46,713 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:46 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/fields/refs HTTP/2.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (2.003 495 2.003) gunicorn-web stdout | 2023-08-08 06:38:46,713 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:46 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/fields/refs HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" exportactionlogsworker stdout | 2023-08-08 06:38:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:38:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:25.514341+00:00 (in 38.115325 seconds) exportactionlogsworker stdout | 2023-08-08 06:38:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:47 UTC)" (scheduled at 2023-08-08 06:38:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:38:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:38:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 47, 399502), True, datetime.datetime(2023, 8, 8, 6, 38, 47, 399502), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:38:47,409 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:38:47,409 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:38:47,409 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:38:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:38:49,178 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:55.371682+00:00 (in 6.192964 seconds) notificationworker stdout | 2023-08-08 06:38:49,178 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:43:49 UTC)" (scheduled at 2023-08-08 06:38:49.178248+00:00) notificationworker stdout | 2023-08-08 06:38:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 38, 49, 179118), 'notification/%']) notificationworker stdout | 2023-08-08 06:38:49,187 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 38, 49, 179118), True, datetime.datetime(2023, 8, 8, 6, 38, 49, 179118), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:38:49,189 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 38, 49, 179118), True, datetime.datetime(2023, 8, 8, 6, 38, 49, 179118), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 38, 49, 179118), 'notification/%']) notificationworker stdout | 2023-08-08 06:38:49,191 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:38:49,191 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:43:49 UTC)" executed successfully gcworker stdout | 2023-08-08 06:38:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:38:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:59.417660+00:00 (in 9.999495 seconds) gcworker stdout | 2023-08-08 06:38:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:59 UTC)" (scheduled at 2023-08-08 06:38:49.417660+00:00) gcworker stdout | 2023-08-08 06:38:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267129418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:38:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:38:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:38:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:38:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:38:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:38:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:38:56.890741+00:00 (in 7.202858 seconds) securityworker stdout | 2023-08-08 06:38:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:19 UTC)" (scheduled at 2023-08-08 06:38:49.687363+00:00) securityworker stdout | 2023-08-08 06:38:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:38:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:38:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:38:49,691 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:38:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:38:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:38:49,762 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:38:50,774 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:38:51,564 [197] [DEBUG] [app] Starting request: urn:request:77683e21-309c-43de-b77c-a6ab4e2be3d1 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:51,565 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:51,565 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:51,565 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:51,573 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:51,573 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:51,573 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:51,573 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:51,573 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:51,573 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:51,574 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:51,574 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:51,577 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:51,583 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:51,584 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:51,587 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:51,591 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:51,596 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:51,597 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:51,599 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:51,599 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:51,599 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:51,601 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:51,811 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None manifestbackfillworker stdout | 2023-08-08 06:38:51,949 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:38:52,109 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:52,112 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:52,178 [199] [DEBUG] [app] Starting request: urn:request:f4ae337a-5c79-448f-96f2-95b874c48bf6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:38:52,179 [200] [DEBUG] [app] Starting request: urn:request:353eda0d-5983-42fd-acc6-8d39dec81118 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:38:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:52,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:38:52,184 [210] [DEBUG] [app] Starting request: urn:request:d12cc09b-e713-44ff-ac58-7c51a8b33db1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:38:52,185 [207] [DEBUG] [app] Starting request: urn:request:f56041f6-2b4b-4058-99ce-578efd802d3f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:38:52,185 [210] [DEBUG] [app] Ending request: urn:request:d12cc09b-e713-44ff-ac58-7c51a8b33db1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d12cc09b-e713-44ff-ac58-7c51a8b33db1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:38:52,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:38:52,185 [207] [DEBUG] [app] Ending request: urn:request:f56041f6-2b4b-4058-99ce-578efd802d3f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f56041f6-2b4b-4058-99ce-578efd802d3f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:38:52,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:38:52,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:38:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:52,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:38:52,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:38:52,190 [197] [DEBUG] [app] Starting request: urn:request:666b9d9e-07e7-4a94-b98f-6d7ba420b63f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:38:52,191 [198] [DEBUG] [app] Starting request: urn:request:30cbe553-9b02-407e-9650-1297cfb9e2e3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:38:52,191 [197] [DEBUG] [app] Ending request: urn:request:666b9d9e-07e7-4a94-b98f-6d7ba420b63f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:666b9d9e-07e7-4a94-b98f-6d7ba420b63f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:38:52,191 [198] [DEBUG] [app] Ending request: urn:request:30cbe553-9b02-407e-9650-1297cfb9e2e3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:30cbe553-9b02-407e-9650-1297cfb9e2e3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:38:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:38:52,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:38:52,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:38:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:38:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:38:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:38:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:38:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:38:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:38:52,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:38:52,203 [200] [DEBUG] [app] Ending request: urn:request:353eda0d-5983-42fd-acc6-8d39dec81118 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:353eda0d-5983-42fd-acc6-8d39dec81118', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:38:52,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:52,203 [199] [DEBUG] [app] Ending request: urn:request:f4ae337a-5c79-448f-96f2-95b874c48bf6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f4ae337a-5c79-448f-96f2-95b874c48bf6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:38:52,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:38:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:38:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:38:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:38:52,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:38:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:38:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:38:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:02.293757+00:00 (in 9.999521 seconds) chunkcleanupworker stdout | 2023-08-08 06:38:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:02 UTC)" (scheduled at 2023-08-08 06:38:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:38:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:38:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 52, 294688), True, datetime.datetime(2023, 8, 8, 6, 38, 52, 294688), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:38:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:38:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:38:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:02 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:52,370 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None storagereplication stdout | 2023-08-08 06:38:52,435 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gunicorn-web stdout | 2023-08-08 06:38:52,670 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:52,863 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:53,069 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/files/Dockerfile?ref=main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:53,071 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:38:53,074 [197] [DEBUG] [app] Ending request: urn:request:77683e21-309c-43de-b77c-a6ab4e2be3d1 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:77683e21-309c-43de-b77c-a6ab4e2be3d1', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'h4051/quay_build_test', 'dockerfile_path': '', 'context': None}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:53,075 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:53,076 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:53 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze HTTP/1.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:53 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze HTTP/2.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.512 539 1.513) securityscanningnotificationworker stdout | 2023-08-08 06:38:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:38:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:13.097375+00:00 (in 18.812775 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:38:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:54 UTC)" (scheduled at 2023-08-08 06:38:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:38:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:38:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:38:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:38:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:36.579841+00:00 (in 42.185699 seconds) namespacegcworker stdout | 2023-08-08 06:38:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:54 UTC)" (scheduled at 2023-08-08 06:38:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:38:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:38:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 54, 394695), True, datetime.datetime(2023, 8, 8, 6, 38, 54, 394695), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:38:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:38:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:38:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:39:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:38:55,074 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-web stdout | 2023-08-08 06:38:55,286 [197] [DEBUG] [app] Starting request: urn:request:619bf90e-643e-410a-8708-fdfc76645497 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:55,286 [197] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:55,286 [197] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:55,287 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:55,293 [197] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:55,293 [197] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:55,293 [197] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:55,294 [197] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:55,294 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:55,294 [197] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:55,294 [197] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:55,294 [197] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:55,296 [197] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:55,302 [197] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:55,303 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:55,306 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:55,309 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:55,311 [197] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:55,312 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:55,314 [197] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:55,314 [197] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:55,314 [197] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:55,316 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 notificationworker stdout | 2023-08-08 06:38:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:38:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:02.178530+00:00 (in 6.806398 seconds) notificationworker stdout | 2023-08-08 06:38:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:05 UTC)" (scheduled at 2023-08-08 06:38:55.371682+00:00) notificationworker stdout | 2023-08-08 06:38:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:38:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 38, 55, 372590), True, datetime.datetime(2023, 8, 8, 6, 38, 55, 372590), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:38:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:38:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:38:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:05 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:55,529 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-secscan stdout | 2023-08-08 06:38:55,552 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:38:55,556 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:38:55,559 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} gunicorn-web stdout | 2023-08-08 06:38:55,877 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:55,880 [197] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:56,064 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None builder stdout | 2023-08-08 06:38:56,292 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:38:56,301 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:38:56,301 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:38:56,305 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:56,333 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:56,520 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:56,780 [197] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/files/Dockerfile?ref=main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:56,782 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:38:56,784 [197] [DEBUG] [app] Ending request: urn:request:619bf90e-643e-410a-8708-fdfc76645497 (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:619bf90e-643e-410a-8708-fdfc76645497', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'h4051/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': None}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:56,785 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:56 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze HTTP/2.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.501 570 1.502) gunicorn-web stdout | 2023-08-08 06:38:56,786 [197] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:56 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze HTTP/1.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" securityworker stdout | 2023-08-08 06:38:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:38:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:19.687363+00:00 (in 22.796132 seconds) securityworker stdout | 2023-08-08 06:38:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:26 UTC)" (scheduled at 2023-08-08 06:38:56.890741+00:00) securityworker stdout | 2023-08-08 06:38:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:38:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:38:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:38:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:38:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:38:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:38:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:38:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:38:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:38:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:38:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 33, 56, 896483), 1, 2]) securityworker stdout | 2023-08-08 06:38:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 33, 56, 896483), 1, 2]) securityworker stdout | 2023-08-08 06:38:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:38:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:38:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:38:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:38:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:38:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:38:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:38:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:38:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:38:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:26 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:38:56,976 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:38:57,336 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:38:57,340 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:38:57,946 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:38:58,075 [200] [DEBUG] [app] Starting request: urn:request:87c1044a-a0f3-497b-aed0-d7f9a1dcc52f (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:38:58,075 [200] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:58,075 [200] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:58,076 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:58,082 [200] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:38:58,083 [200] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:38:58,083 [200] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:58,083 [200] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:38:58,083 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:58,083 [200] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:58,083 [200] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:58,083 [200] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:38:58,085 [200] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:38:58,092 [200] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:38:58,093 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:58,095 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:58,098 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:38:58,101 [200] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:38:58,102 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [3, True]) gunicorn-web stdout | 2023-08-08 06:38:58,104 [200] [DEBUG] [auth.permissions] Organization team added permission: _NamespaceWideNeed(type='organization', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:58,104 [200] [DEBUG] [auth.permissions] Organization team added repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1_build_test', role='admin') gunicorn-web stdout | 2023-08-08 06:38:58,104 [200] [DEBUG] [auth.permissions] Team added permission: _TeamTypeNeed(type='orgteam', orgname='user1_build_test', teamname='owners', role='admin') gunicorn-web stdout | 2023-08-08 06:38:58,109 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:58,244 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:38:58,305 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:58,362 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:38:58,376 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:38:58,381 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} teamsyncworker stdout | 2023-08-08 06:38:58,383 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} gunicorn-web stdout | 2023-08-08 06:38:58,592 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:58,595 [200] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:38:58,810 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None repositoryactioncounter stdout | 2023-08-08 06:38:58,886 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:38:59,077 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gunicorn-web stdout | 2023-08-08 06:38:59,378 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None gcworker stdout | 2023-08-08 06:38:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:38:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:09.417660+00:00 (in 9.999437 seconds) gcworker stdout | 2023-08-08 06:38:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:09 UTC)" (scheduled at 2023-08-08 06:38:59.417660+00:00) gcworker stdout | 2023-08-08 06:38:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267139418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:38:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:38:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:38:59,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:09 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:38:59,562 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/branches HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:59,875 [200] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/35911436/repository/files/Dockerfile?ref=main HTTP/1.1" 200 None gunicorn-web stdout | 2023-08-08 06:38:59,876 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed", "t2"."id", "t2"."robot_account_id", "t2"."token", "t2"."fully_migrated", "t3"."id", "t3"."robot_account_id", "t3"."description", "t3"."unstructured_json" FROM "user" AS "t1" LEFT OUTER JOIN "robotaccountmetadata" AS "t3" ON ("t3"."robot_account_id" = "t1"."id") INNER JOIN "robotaccounttoken" AS "t2" ON ("t2"."robot_account_id" = "t1"."id") WHERE (("t1"."robot" = %s) AND ("t1"."username" ILIKE %s))', [True, 'user1_build_test+%']) gunicorn-web stdout | 2023-08-08 06:38:59,879 [200] [DEBUG] [app] Ending request: urn:request:87c1044a-a0f3-497b-aed0-d7f9a1dcc52f (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze) {'endpoint': 'api.buildtriggeranalyze', 'request_id': 'urn:request:87c1044a-a0f3-497b-aed0-d7f9a1dcc52f', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze', 'parameters': {}, 'json_body': {'config': {'build_source': 'h4051/quay_build_test', 'dockerfile_path': 'Dockerfile', 'context': '/'}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:38:59,880 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:38:59,880 [200] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:38:59 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze HTTP/1.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:38:59 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/analyze HTTP/2.0" 200 121 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.807 774 1.806) buildlogsarchiver stdout | 2023-08-08 06:39:00,453 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:39:00,640 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:39:01,624 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:39:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:39:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:05.371682+00:00 (in 3.192627 seconds) notificationworker stdout | 2023-08-08 06:39:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:02 UTC)" (scheduled at 2023-08-08 06:39:02.178530+00:00) notificationworker stdout | 2023-08-08 06:39:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:39:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:39:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:39:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:12.293757+00:00 (in 9.999514 seconds) chunkcleanupworker stdout | 2023-08-08 06:39:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:12 UTC)" (scheduled at 2023-08-08 06:39:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:39:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:39:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 2, 294800), True, datetime.datetime(2023, 8, 8, 6, 39, 2, 294800), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:39:02,304 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:39:02,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:39:02,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:39:02,504 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:39:02,637 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:39:02,995 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} gunicorn-web stdout | 2023-08-08 06:39:03,047 [199] [DEBUG] [app] Starting request: urn:request:b7fd9999-43a6-43e7-8145-e7beabdb59cb (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/activate) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:39:03,047 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:39:03,047 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:39:03,048 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:03,055 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:39:03,055 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:39:03,055 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:39:03,055 [199] [DEBUG] [endpoints.api] Checking permission for repo: user1_build_test/gitlab gunicorn-web stdout | 2023-08-08 06:39:03,055 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:39:03,055 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:39:03,055 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:39:03,055 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:39:03,058 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['gitlab', 'user1_build_test', 3, 'gitlab', 'user1_build_test', 3]) gunicorn-web stdout | 2023-08-08 06:39:03,064 [199] [DEBUG] [auth.permissions] User added permission: _ResourceNeed(type='repository', namespace='user1_build_test', name='gitlab', role='admin') gunicorn-web stdout | 2023-08-08 06:39:03,064 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 'application', 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:03,067 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:03,070 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."service_id", "t1"."repository_id", "t1"."connected_user_id", "t1"."secure_auth_token", "t1"."secure_private_key", "t1"."fully_migrated", "t1"."config", "t1"."write_token_id", "t1"."pull_robot_id", "t1"."enabled", "t1"."disabled_reason_id", "t1"."disabled_datetime", "t1"."successive_failure_count", "t1"."successive_internal_error_count", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorybuildtrigger" AS "t1" INNER JOIN "buildtriggerservice" AS "t2" ON ("t1"."service_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."connected_user_id" = "t5"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['879924fa-0bc9-4217-93fe-9bfc73feaf0e', 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:03,073 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:03,074 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:39:03,074 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['write', 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:03,076 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['user1_build_test', 'gitlab', 3, 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:03,077 [199] [DEBUG] [peewee] ('INSERT INTO "accesstoken" ("friendly_name", "token_name", "token_code", "repository_id", "created", "role_id", "temporary") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "accesstoken"."id"', ['Build Trigger: gitlab', 'R9LDR0GO5857ZCFRE2EBS34DJYNR97KV', 'v0$$JDQLEE2cRRZlVEMjNXdsGfl2uR/i89x/hJBODT6/bLkDlfZXf8uoQcSWyf6i4zsspqTJVUbWL5BktKZAHA==', 5, datetime.datetime(2023, 8, 8, 6, 39, 3, 77444), 2, False]) gunicorn-web stdout | 2023-08-08 06:39:03,083 [199] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): gitlab.com:443 gunicorn-web stdout | 2023-08-08 06:39:03,291 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/user HTTP/1.1" 200 None securityscanningnotificationworker stdout | 2023-08-08 06:39:03,538 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-web stdout | 2023-08-08 06:39:03,592 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "GET /api/v4/projects/h4051%2Fquay_build_test HTTP/1.1" 200 None namespacegcworker stdout | 2023-08-08 06:39:03,849 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-web stdout | 2023-08-08 06:39:03,867 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "POST /api/v4/projects/35911436/deploy_keys HTTP/1.1" 201 673 gunicorn-registry stdout | 2023-08-08 06:39:03,920 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:39:03,923 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:39:03,926 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:39:03,929 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:39:03,932 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:39:03,936 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:39:03,940 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:39:03,943 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:39:03,947 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} gunicorn-web stdout | 2023-08-08 06:39:04,112 [199] [DEBUG] [urllib3.connectionpool] https://gitlab.com:443 "POST /api/v4/projects/35911436/hooks HTTP/1.1" 422 29 gunicorn-web stdout | 2023-08-08 06:39:04,113 [199] [ERROR] [buildtrigger.gitlabhandler] GitLab API error. Please contact support. gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 267, in wrapped_f gunicorn-web stdout | return f(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/mixins.py", line 211, in create gunicorn-web stdout | server_data = self.gitlab.http_post(path, post_data=data, files=files, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 680, in http_post gunicorn-web stdout | result = self.http_request( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 573, in http_request gunicorn-web stdout | raise GitlabHttpError( gunicorn-web stdout | gitlab.exceptions.GitlabHttpError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 100, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 270, in activate gunicorn-web stdout | hook = gl_project.hooks.create( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 269, in wrapped_f gunicorn-web stdout | raise error(e.error_message, e.response_code, e.response_body) gunicorn-web stdout | gitlab.exceptions.GitlabCreateError: 422: Invalid url given gunicorn-web stdout | 2023-08-08 06:39:04,114 [199] [ERROR] [app] Exception on /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/activate [POST] gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 267, in wrapped_f gunicorn-web stdout | return f(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/mixins.py", line 211, in create gunicorn-web stdout | server_data = self.gitlab.http_post(path, post_data=data, files=files, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 680, in http_post gunicorn-web stdout | result = self.http_request( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/__init__.py", line 573, in http_request gunicorn-web stdout | raise GitlabHttpError( gunicorn-web stdout | gitlab.exceptions.GitlabHttpError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 100, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 270, in activate gunicorn-web stdout | hook = gl_project.hooks.create( gunicorn-web stdout | File "/app/lib/python3.9/site-packages/gitlab/exceptions.py", line 269, in wrapped_f gunicorn-web stdout | raise error(e.error_message, e.response_code, e.response_body) gunicorn-web stdout | gitlab.exceptions.GitlabCreateError: 422: Invalid url given gunicorn-web stdout | During handling of the above exception, another exception occurred: gunicorn-web stdout | Traceback (most recent call last): gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/app.py", line 1949, in full_dispatch_request gunicorn-web stdout | rv = self.dispatch_request() gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/app.py", line 1935, in dispatch_request gunicorn-web stdout | return self.view_functions[rule.endpoint](**req.view_args) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 256, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/auth/decorators.py", line 65, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/util/request.py", line 50, in wrapper gunicorn-web stdout | return decorator(func)(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/utils/cors.py", line 35, in wrapped_function gunicorn-web stdout | resp = make_response(f(*args, **kwargs)) gunicorn-web stdout | File "/quay-registry/endpoints/csrf.py", line 73, in wrapper gunicorn-web stdout | resp = func(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/__init__.py", line 467, in wrapper gunicorn-web stdout | resp = resource(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask/views.py", line 89, in view gunicorn-web stdout | return self.dispatch_request(*args, **kwargs) gunicorn-web stdout | File "/app/lib/python3.9/site-packages/flask_restful/__init__.py", line 582, in dispatch_request gunicorn-web stdout | resp = meth(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 189, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 247, in wrapper gunicorn-web stdout | return func(namespace, repository, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 164, in wrapper gunicorn-web stdout | return func(*args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 342, in wrapped gunicorn-web stdout | return func(self, namespace, repository, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 296, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 308, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 283, in wrapped gunicorn-web stdout | return func(self, namespace_name, repository_name, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 494, in wrapped gunicorn-web stdout | return func(self, *args, **kwargs) gunicorn-web stdout | File "/quay-registry/endpoints/api/trigger.py", line 323, in post gunicorn-web stdout | final_config, private_config = handler.activate(authed_url) gunicorn-web stdout | File "/quay-registry/buildtrigger/gitlabhandler.py", line 108, in wrapper gunicorn-web stdout | raise ExternalServiceError(msg) gunicorn-web stdout | endpoints.exception.ExternalServiceError: 520 Unknown Error: GitLab API error. Please contact support. gunicorn-web stdout | 2023-08-08 06:39:04,116 [199] [DEBUG] [app] Ending request: urn:request:b7fd9999-43a6-43e7-8145-e7beabdb59cb (/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/activate) {'endpoint': 'api.buildtriggeractivate', 'request_id': 'urn:request:b7fd9999-43a6-43e7-8145-e7beabdb59cb', 'remote_addr': '10.130.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/activate', 'path': '/api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/activate', 'parameters': {}, 'json_body': {'config': {'build_source': 'h4051/quay_build_test', 'dockerfile_path': '/Dockerfile', 'context': '/', 'default_tag_from_ref': True, 'latest_for_default_branch': True, 'tag_templates': [], 'credentials': [{'name': 'SSH Public Key', 'value': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRGlqBgN09Irt7mo8tE8/L+utq4nk5FmRGS13iyiEdOXSIMSQo2qE6HVmNsaLfLjwOAahP3Ci7MoD3lbv2gdzMSM7ss9dXNfbM7bVtlBByWCLrm2qJPw7XQG7wO5vDsxecxjbIvbhxwEdLQ1rlRvJT/NGMmO8yzSPcduDUtJf695qNkmBSRed5kyrRf4DBIyVxl9G80btribNi30YP13WFH7Vl57BB0cfVDBvHv2jTG9AkyP43klfrDR98dGa5FSmLbVHm4l6rgtI8v54YE2Ec31/XJ9f92J+JHc3aODx9RNTKqky53yP8XabG2uzC10mp8MSgZZJ/9pNMNE+WYWIp'}], 'key_id': 12098911}}, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:39:04,117 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:39:04,117 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:39:04 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/activate HTTP/1.0" 520 348 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:39:04 +0000] "POST /api/v1/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e/activate HTTP/2.0" 520 348 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (1.071 851 1.071) repositorygcworker stdout | 2023-08-08 06:39:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:39:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:09.793658+00:00 (in 5.605742 seconds) repositorygcworker stdout | 2023-08-08 06:39:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:04 UTC)" (scheduled at 2023-08-08 06:39:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:39:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:39:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:04 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:39:04,337 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:39:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:39:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:15.371682+00:00 (in 9.999518 seconds) notificationworker stdout | 2023-08-08 06:39:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:15 UTC)" (scheduled at 2023-08-08 06:39:05.371682+00:00) notificationworker stdout | 2023-08-08 06:39:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:39:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 5, 372604), True, datetime.datetime(2023, 8, 8, 6, 39, 5, 372604), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:39:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:39:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:39:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:15 UTC)" executed successfully builder stdout | 2023-08-08 06:39:06,316 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:39:06,325 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:39:06,325 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:39:06,330 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:39:07,179 [197] [DEBUG] [app] Starting request: urn:request:6fbd6a3a-6966-4b89-9044-290f95df14a0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:39:07,179 [200] [DEBUG] [app] Starting request: urn:request:32af522e-3b15-4003-b746-68448ac0ef06 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:39:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:07,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:39:07,185 [203] [DEBUG] [app] Starting request: urn:request:b334b5c5-5aee-44fe-86b7-2f74696187bd (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:39:07,185 [203] [DEBUG] [app] Ending request: urn:request:b334b5c5-5aee-44fe-86b7-2f74696187bd (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b334b5c5-5aee-44fe-86b7-2f74696187bd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:39:07,186 [210] [DEBUG] [app] Starting request: urn:request:b9214fb8-eb84-43b2-bf09-46010eb7bd02 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:39:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:39:07,186 [210] [DEBUG] [app] Ending request: urn:request:b9214fb8-eb84-43b2-bf09-46010eb7bd02 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b9214fb8-eb84-43b2-bf09-46010eb7bd02', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:39:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:07,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:07,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:07,191 [200] [DEBUG] [app] Starting request: urn:request:5c5a4f02-adbc-4655-a822-87a684b4be19 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:39:07,191 [200] [DEBUG] [app] Ending request: urn:request:5c5a4f02-adbc-4655-a822-87a684b4be19 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5c5a4f02-adbc-4655-a822-87a684b4be19', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:39:07,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:39:07,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:07,193 [199] [DEBUG] [app] Starting request: urn:request:4356aa78-dfb5-46c8-af03-45d1d9838980 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:39:07,194 [199] [DEBUG] [app] Ending request: urn:request:4356aa78-dfb5-46c8-af03-45d1d9838980 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4356aa78-dfb5-46c8-af03-45d1d9838980', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:39:07,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:07,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:07,195 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:39:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:39:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:39:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:37.197412+00:00 (in 29.999541 seconds) buildlogsarchiver stdout | 2023-08-08 06:39:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:37 UTC)" (scheduled at 2023-08-08 06:39:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:39:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 39, 7, 198674), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:07,199 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:39:07,203 [197] [DEBUG] [app] Ending request: urn:request:6fbd6a3a-6966-4b89-9044-290f95df14a0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6fbd6a3a-6966-4b89-9044-290f95df14a0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:39:07,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:39:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:39:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:39:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:39:07,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:07,205 [200] [DEBUG] [app] Ending request: urn:request:32af522e-3b15-4003-b746-68448ac0ef06 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:32af522e-3b15-4003-b746-68448ac0ef06', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:39:07,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:39:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.028 47 0.027) gunicorn-web stdout | 2023-08-08 06:39:07,206 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:39:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:39:07,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:39:07,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:39:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:39:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:39:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:39:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:19.417660+00:00 (in 9.999487 seconds) gcworker stdout | 2023-08-08 06:39:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:19 UTC)" (scheduled at 2023-08-08 06:39:09.417660+00:00) gcworker stdout | 2023-08-08 06:39:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267149418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:39:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:39:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:39:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:39:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:39:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:19.187230+00:00 (in 9.393099 seconds) repositorygcworker stdout | 2023-08-08 06:39:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:09 UTC)" (scheduled at 2023-08-08 06:39:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:39:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:39:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 9, 794475), True, datetime.datetime(2023, 8, 8, 6, 39, 9, 794475), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:39:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:39:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:39:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:39:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:39:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:22.293757+00:00 (in 9.999442 seconds) chunkcleanupworker stdout | 2023-08-08 06:39:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:22 UTC)" (scheduled at 2023-08-08 06:39:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:39:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:39:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 12, 294822), True, datetime.datetime(2023, 8, 8, 6, 39, 12, 294822), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:39:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:39:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:39:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:39:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:39:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:54.284093+00:00 (in 41.186192 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:39:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:13 UTC)" (scheduled at 2023-08-08 06:39:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:39:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:39:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 13, 98512), True, datetime.datetime(2023, 8, 8, 6, 39, 13, 98512), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:39:13,107 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:39:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:39:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:39:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:39:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:25.371682+00:00 (in 9.999472 seconds) notificationworker stdout | 2023-08-08 06:39:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:25 UTC)" (scheduled at 2023-08-08 06:39:15.371682+00:00) notificationworker stdout | 2023-08-08 06:39:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:39:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 15, 372685), True, datetime.datetime(2023, 8, 8, 6, 39, 15, 372685), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:39:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:39:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:39:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:25 UTC)" executed successfully builder stdout | 2023-08-08 06:39:16,332 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:39:16,341 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:39:16,341 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:39:16,346 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:39:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:39:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:04.187429+00:00 (in 44.999641 seconds) repositorygcworker stdout | 2023-08-08 06:39:19,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:44:19 UTC)" (scheduled at 2023-08-08 06:39:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:39:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 39, 19, 188331), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:39:19,196 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 39, 19, 188331), True, datetime.datetime(2023, 8, 8, 6, 39, 19, 188331), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:39:19,198 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 39, 19, 188331), True, datetime.datetime(2023, 8, 8, 6, 39, 19, 188331), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 39, 19, 188331), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:39:19,200 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:39:19,200 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:44:19 UTC)" executed successfully gcworker stdout | 2023-08-08 06:39:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:39:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:29.417660+00:00 (in 9.999479 seconds) gcworker stdout | 2023-08-08 06:39:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:29 UTC)" (scheduled at 2023-08-08 06:39:19.417660+00:00) gcworker stdout | 2023-08-08 06:39:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267159418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:39:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:39:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:39:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:39:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:39:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:26.890741+00:00 (in 7.202838 seconds) securityworker stdout | 2023-08-08 06:39:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:49 UTC)" (scheduled at 2023-08-08 06:39:19.687363+00:00) securityworker stdout | 2023-08-08 06:39:19,689 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:39:19,689 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:39:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:39:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:39:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:39:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:39:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 34, 19, 692505), 1, 2]) securityworker stdout | 2023-08-08 06:39:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 34, 19, 692505), 1, 2]) securityworker stdout | 2023-08-08 06:39:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:39:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:49 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:39:19,780 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:39:20,812 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:39:21,987 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:39:22,179 [197] [DEBUG] [app] Starting request: urn:request:f8eab8a7-7cce-4b35-ad1d-7f9b973b66dd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:39:22,180 [200] [DEBUG] [app] Starting request: urn:request:902c9325-ccd2-4e6b-9445-3da9c7d3cfac (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:39:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:22,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:39:22,185 [203] [DEBUG] [app] Starting request: urn:request:e57c5fcf-4d77-4769-b58d-63551a83cde9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:39:22,185 [210] [DEBUG] [app] Starting request: urn:request:50c685bb-5193-416a-9b54-bf9a4c52f6ef (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:39:22,186 [203] [DEBUG] [app] Ending request: urn:request:e57c5fcf-4d77-4769-b58d-63551a83cde9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e57c5fcf-4d77-4769-b58d-63551a83cde9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:39:22,186 [210] [DEBUG] [app] Ending request: urn:request:50c685bb-5193-416a-9b54-bf9a4c52f6ef (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:50c685bb-5193-416a-9b54-bf9a4c52f6ef', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:39:22,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:39:22,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:39:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:22,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:22,192 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:22,192 [197] [DEBUG] [app] Starting request: urn:request:8ca1914b-372c-4264-9e8b-f5c54d73ea18 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:39:22,192 [197] [DEBUG] [app] Ending request: urn:request:8ca1914b-372c-4264-9e8b-f5c54d73ea18 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8ca1914b-372c-4264-9e8b-f5c54d73ea18', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:39:22,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:39:22,193 [199] [DEBUG] [app] Starting request: urn:request:464646a3-92e9-46c8-8426-1bd136ee36f1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:39:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:39:22,193 [199] [DEBUG] [app] Ending request: urn:request:464646a3-92e9-46c8-8426-1bd136ee36f1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:464646a3-92e9-46c8-8426-1bd136ee36f1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:39:22,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:22,194 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:22,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:39:22,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:39:22,200 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:39:22,204 [197] [DEBUG] [app] Ending request: urn:request:f8eab8a7-7cce-4b35-ad1d-7f9b973b66dd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f8eab8a7-7cce-4b35-ad1d-7f9b973b66dd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:39:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:39:22,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:39:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:39:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:39:22,205 [200] [DEBUG] [app] Ending request: urn:request:902c9325-ccd2-4e6b-9445-3da9c7d3cfac (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:902c9325-ccd2-4e6b-9445-3da9c7d3cfac', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:39:22,205 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:39:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:39:22,205 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:39:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:39:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:39:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:28.874906+00:00 (in 6.580539 seconds) chunkcleanupworker stdout | 2023-08-08 06:39:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:32 UTC)" (scheduled at 2023-08-08 06:39:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:39:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:39:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 22, 294962), True, datetime.datetime(2023, 8, 8, 6, 39, 22, 294962), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:39:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:39:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:39:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:32 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:39:22,473 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gcworker stdout | 2023-08-08 06:39:25,111 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:39:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:39:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:35.371682+00:00 (in 9.999493 seconds) notificationworker stdout | 2023-08-08 06:39:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:35 UTC)" (scheduled at 2023-08-08 06:39:25.371682+00:00) notificationworker stdout | 2023-08-08 06:39:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:39:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 25, 372753), True, datetime.datetime(2023, 8, 8, 6, 39, 25, 372753), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:39:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:39:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:39:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:39:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:39:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:47.398551+00:00 (in 21.883687 seconds) exportactionlogsworker stdout | 2023-08-08 06:39:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:25 UTC)" (scheduled at 2023-08-08 06:39:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:39:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:39:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:25 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:39:25,561 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:39:25,565 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:39:25,568 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:39:26,346 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:39:26,355 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:39:26,355 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:39:26,359 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:39:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:39:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:49.687363+00:00 (in 22.795952 seconds) securityworker stdout | 2023-08-08 06:39:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:56 UTC)" (scheduled at 2023-08-08 06:39:26.890741+00:00) securityworker stdout | 2023-08-08 06:39:26,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:39:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:39:26,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:39:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:39:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:39:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:39:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:39:26,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:39:26,898 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:39:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:39:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 34, 26, 897816), 1, 2]) securityworker stdout | 2023-08-08 06:39:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:26,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 34, 26, 897816), 1, 2]) securityworker stdout | 2023-08-08 06:39:26,911 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:26,911 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:26,911 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:39:26,911 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:39:26,911 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:39:26,911 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:39:26,911 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:39:26,912 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:39:26,912 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:39:26,912 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:39:56 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:39:26,989 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:39:27,344 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:39:27,377 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:39:27,954 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:39:28,256 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:39:28,377 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:39:28,394 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:39:28,398 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} teamsyncworker stdout | 2023-08-08 06:39:28,409 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} chunkcleanupworker stdout | 2023-08-08 06:39:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:39:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:32.293757+00:00 (in 3.418204 seconds) chunkcleanupworker stdout | 2023-08-08 06:39:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:28 UTC)" (scheduled at 2023-08-08 06:39:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:39:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:39:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:28 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 06:39:28,924 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:39:29,114 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:39:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:39:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:39.417660+00:00 (in 9.999440 seconds) gcworker stdout | 2023-08-08 06:39:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:39 UTC)" (scheduled at 2023-08-08 06:39:29.417660+00:00) gcworker stdout | 2023-08-08 06:39:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267169418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:39:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:39:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:39:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:39:30,491 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:39:30,675 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:39:31,662 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:39:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:39:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:42.293757+00:00 (in 9.999433 seconds) chunkcleanupworker stdout | 2023-08-08 06:39:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:42 UTC)" (scheduled at 2023-08-08 06:39:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:39:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:39:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 32, 294703), True, datetime.datetime(2023, 8, 8, 6, 39, 32, 294703), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:39:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:39:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:39:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:39:32,514 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:39:32,650 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:39:33,004 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:39:33,574 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:39:33,887 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:39:33,928 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:39:33,932 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:39:33,934 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:39:33,937 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:39:33,941 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:39:33,945 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:39:33,950 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:39:33,952 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:39:33,955 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:39:34,374 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:39:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:39:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:45.371682+00:00 (in 9.999446 seconds) notificationworker stdout | 2023-08-08 06:39:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:45 UTC)" (scheduled at 2023-08-08 06:39:35.371682+00:00) notificationworker stdout | 2023-08-08 06:39:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:39:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 35, 372715), True, datetime.datetime(2023, 8, 8, 6, 39, 35, 372715), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:39:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:39:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:39:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:45 UTC)" executed successfully builder stdout | 2023-08-08 06:39:36,361 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:39:36,369 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:39:36,369 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:39:36,375 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:39:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:39:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:39.579576+00:00 (in 2.999105 seconds) namespacegcworker stdout | 2023-08-08 06:39:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:36 UTC)" (scheduled at 2023-08-08 06:39:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:39:36,581 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:39:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:39:37,179 [197] [DEBUG] [app] Starting request: urn:request:6d7c85f3-add9-4413-a084-85ec87ea2f96 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:39:37,180 [200] [DEBUG] [app] Starting request: urn:request:a03d1bea-75e1-48dd-b9eb-b0cabff8deda (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:39:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:37,185 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:39:37,186 [209] [DEBUG] [app] Starting request: urn:request:1ee7f007-8149-4d1c-9418-8e8c1f8a9875 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:39:37,186 [209] [DEBUG] [app] Ending request: urn:request:1ee7f007-8149-4d1c-9418-8e8c1f8a9875 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1ee7f007-8149-4d1c-9418-8e8c1f8a9875', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:39:37,186 [205] [DEBUG] [app] Starting request: urn:request:c511e477-106f-4d26-8732-25afe9048121 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:39:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:37,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:39:37,187 [205] [DEBUG] [app] Ending request: urn:request:c511e477-106f-4d26-8732-25afe9048121 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c511e477-106f-4d26-8732-25afe9048121', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:39:37,187 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:37,187 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:37,189 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:37,189 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:37,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:37,192 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:37,192 [200] [DEBUG] [app] Starting request: urn:request:35146197-eba5-4415-ace2-670f11353709 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:39:37,193 [200] [DEBUG] [app] Ending request: urn:request:35146197-eba5-4415-ace2-670f11353709 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:35146197-eba5-4415-ace2-670f11353709', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:39:37,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:37,193 [199] [DEBUG] [app] Starting request: urn:request:79df7a6f-dcd7-45b6-8be4-3fa257a45767 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:39:37,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:37,194 [199] [DEBUG] [app] Ending request: urn:request:79df7a6f-dcd7-45b6-8be4-3fa257a45767 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:79df7a6f-dcd7-45b6-8be4-3fa257a45767', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:39:37,194 [200] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:39:37,194 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:37,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:37,195 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:39:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:39:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:07.197412+00:00 (in 29.999367 seconds) buildlogsarchiver stdout | 2023-08-08 06:39:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:07 UTC)" (scheduled at 2023-08-08 06:39:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:39:37,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:39:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 39, 37, 198528), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:37,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:39:37,204 [200] [DEBUG] [app] Ending request: urn:request:a03d1bea-75e1-48dd-b9eb-b0cabff8deda (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a03d1bea-75e1-48dd-b9eb-b0cabff8deda', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:39:37,204 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:39:37,204 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:39:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:39:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:39:37,204 [197] [DEBUG] [app] Ending request: urn:request:6d7c85f3-add9-4413-a084-85ec87ea2f96 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6d7c85f3-add9-4413-a084-85ec87ea2f96', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:39:37,205 [197] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:39:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2023-08-08 06:39:37,205 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:39:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:39:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:39:37,206 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:39:37,207 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:39:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:39:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:39:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:39:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:49.417660+00:00 (in 9.999483 seconds) gcworker stdout | 2023-08-08 06:39:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:49 UTC)" (scheduled at 2023-08-08 06:39:39.417660+00:00) gcworker stdout | 2023-08-08 06:39:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267179418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:39:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:39:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:39:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:39:39,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:39:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:54.393627+00:00 (in 14.813386 seconds) namespacegcworker stdout | 2023-08-08 06:39:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:44:39 UTC)" (scheduled at 2023-08-08 06:39:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:39:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 39, 39, 580577), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:39:39,587 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 39, 39, 580577), True, datetime.datetime(2023, 8, 8, 6, 39, 39, 580577), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:39:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 39, 39, 580577), True, datetime.datetime(2023, 8, 8, 6, 39, 39, 580577), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 39, 39, 580577), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:39:39,590 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:39:39,590 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:44:39 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:39:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:39:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:46.874674+00:00 (in 4.580437 seconds) chunkcleanupworker stdout | 2023-08-08 06:39:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:52 UTC)" (scheduled at 2023-08-08 06:39:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:39:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:39:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 42, 294564), True, datetime.datetime(2023, 8, 8, 6, 39, 42, 294564), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:39:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:39:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:39:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:39:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:39:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:55.371682+00:00 (in 9.999499 seconds) notificationworker stdout | 2023-08-08 06:39:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:55 UTC)" (scheduled at 2023-08-08 06:39:45.371682+00:00) notificationworker stdout | 2023-08-08 06:39:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:39:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 45, 372557), True, datetime.datetime(2023, 8, 8, 6, 39, 45, 372557), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:39:45,383 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:39:45,383 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:39:45,383 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:55 UTC)" executed successfully builder stdout | 2023-08-08 06:39:46,375 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:39:46,384 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:39:46,384 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:39:46,389 [55] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:39:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:39:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:52.293757+00:00 (in 5.418597 seconds) chunkcleanupworker stdout | 2023-08-08 06:39:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:44:46 UTC)" (scheduled at 2023-08-08 06:39:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:39:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 39, 46, 875484), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:39:46,882 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 39, 46, 875484), True, datetime.datetime(2023, 8, 8, 6, 39, 46, 875484), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:39:46,884 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 39, 46, 875484), True, datetime.datetime(2023, 8, 8, 6, 39, 46, 875484), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 39, 46, 875484), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:39:46,885 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:39:46,885 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:44:46 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:39:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:39:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:25.514341+00:00 (in 38.115302 seconds) exportactionlogsworker stdout | 2023-08-08 06:39:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:47 UTC)" (scheduled at 2023-08-08 06:39:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:39:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:39:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 47, 399508), True, datetime.datetime(2023, 8, 8, 6, 39, 47, 399508), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:39:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:39:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:39:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:39:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:39:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:59.417660+00:00 (in 9.999465 seconds) gcworker stdout | 2023-08-08 06:39:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:59 UTC)" (scheduled at 2023-08-08 06:39:49.417660+00:00) gcworker stdout | 2023-08-08 06:39:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267189418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:39:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:39:49,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:39:49,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:39:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:39:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:39:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:39:56.890741+00:00 (in 7.202864 seconds) securityworker stdout | 2023-08-08 06:39:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:19 UTC)" (scheduled at 2023-08-08 06:39:49.687363+00:00) securityworker stdout | 2023-08-08 06:39:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:39:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:39:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:39:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:39:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:39:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:39:49,794 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:39:50,849 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:39:52,003 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:39:52,179 [198] [DEBUG] [app] Starting request: urn:request:a29933ee-0aeb-4cb9-88fb-5a77cfea5b70 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:39:52,179 [197] [DEBUG] [app] Starting request: urn:request:404ae5d4-c820-4f94-8bba-1349d83b9610 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:39:52,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:52,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:39:52,184 [203] [DEBUG] [app] Starting request: urn:request:017451b9-d362-45cd-b856-ad6ce5492afd (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:39:52,185 [207] [DEBUG] [app] Starting request: urn:request:214c9d08-16b6-4ead-879e-0e625409c536 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:39:52,185 [203] [DEBUG] [app] Ending request: urn:request:017451b9-d362-45cd-b856-ad6ce5492afd (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:017451b9-d362-45cd-b856-ad6ce5492afd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:39:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:39:52,185 [207] [DEBUG] [app] Ending request: urn:request:214c9d08-16b6-4ead-879e-0e625409c536 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:214c9d08-16b6-4ead-879e-0e625409c536', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:39:52,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:39:52,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:52,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:39:52,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:39:52,190 [200] [DEBUG] [app] Starting request: urn:request:fd7e2563-25cd-424b-b978-c4fbdedbc6b1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:39:52,191 [200] [DEBUG] [app] Ending request: urn:request:fd7e2563-25cd-424b-b978-c4fbdedbc6b1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fd7e2563-25cd-424b-b978-c4fbdedbc6b1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:39:52,191 [199] [DEBUG] [app] Starting request: urn:request:3a945086-7f71-4c43-8110-d2ad76afd7ed (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:39:52,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:39:52,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:52,191 [199] [DEBUG] [app] Ending request: urn:request:3a945086-7f71-4c43-8110-d2ad76afd7ed (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3a945086-7f71-4c43-8110-d2ad76afd7ed', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:39:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:39:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:39:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:39:52,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:39:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:39:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:39:52,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:39:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:39:52,203 [198] [DEBUG] [app] Ending request: urn:request:a29933ee-0aeb-4cb9-88fb-5a77cfea5b70 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a29933ee-0aeb-4cb9-88fb-5a77cfea5b70', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:39:52,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:39:52,203 [197] [DEBUG] [app] Ending request: urn:request:404ae5d4-c820-4f94-8bba-1349d83b9610 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:404ae5d4-c820-4f94-8bba-1349d83b9610', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:39:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:39:52,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:39:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:39:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:39:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:39:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:39:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) chunkcleanupworker stdout | 2023-08-08 06:39:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:39:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:02.293757+00:00 (in 9.999531 seconds) chunkcleanupworker stdout | 2023-08-08 06:39:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:02 UTC)" (scheduled at 2023-08-08 06:39:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:39:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:39:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 52, 294550), True, datetime.datetime(2023, 8, 8, 6, 39, 52, 294550), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:39:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:39:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:39:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:02 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:39:52,489 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:39:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:39:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:13.097375+00:00 (in 18.812688 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:39:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:54 UTC)" (scheduled at 2023-08-08 06:39:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:39:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:39:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:39:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:39:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:36.579841+00:00 (in 42.185720 seconds) namespacegcworker stdout | 2023-08-08 06:39:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:54 UTC)" (scheduled at 2023-08-08 06:39:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:39:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:39:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 54, 394486), True, datetime.datetime(2023, 8, 8, 6, 39, 54, 394486), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:39:54,403 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:39:54,403 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:39:54,403 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:40:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:39:55,149 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:39:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:39:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:02.178530+00:00 (in 6.806370 seconds) notificationworker stdout | 2023-08-08 06:39:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:05 UTC)" (scheduled at 2023-08-08 06:39:55.371682+00:00) notificationworker stdout | 2023-08-08 06:39:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:39:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 39, 55, 372681), True, datetime.datetime(2023, 8, 8, 6, 39, 55, 372681), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:39:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:39:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:39:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:39:55,570 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:39:55,573 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:39:55,577 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:39:56,399 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:39:56,408 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:39:56,408 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:39:56,412 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:39:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:39:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:19.687363+00:00 (in 22.796130 seconds) securityworker stdout | 2023-08-08 06:39:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:26 UTC)" (scheduled at 2023-08-08 06:39:56.890741+00:00) securityworker stdout | 2023-08-08 06:39:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:39:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:39:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:39:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:39:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:39:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:39:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:39:56,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:39:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:39:56,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:39:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 34, 56, 896190), 1, 2]) securityworker stdout | 2023-08-08 06:39:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 34, 56, 896190), 1, 2]) securityworker stdout | 2023-08-08 06:39:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:39:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:39:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:39:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:39:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:39:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:39:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:39:56,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:39:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:39:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:26 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:39:57,026 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:39:57,381 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:39:57,414 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:39:57,962 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:39:58,269 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:39:58,391 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:39:58,408 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:39:58,413 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} teamsyncworker stdout | 2023-08-08 06:39:58,445 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} repositoryactioncounter stdout | 2023-08-08 06:39:58,953 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:39:59,145 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:39:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:39:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:09.417660+00:00 (in 9.999475 seconds) gcworker stdout | 2023-08-08 06:39:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:09 UTC)" (scheduled at 2023-08-08 06:39:59.417660+00:00) gcworker stdout | 2023-08-08 06:39:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267199418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:39:59,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:39:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:39:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:40:00,528 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:40:00,685 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:40:01,700 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:40:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:40:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:05.371682+00:00 (in 3.192637 seconds) notificationworker stdout | 2023-08-08 06:40:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:02 UTC)" (scheduled at 2023-08-08 06:40:02.178530+00:00) notificationworker stdout | 2023-08-08 06:40:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:40:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:40:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:40:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:12.293757+00:00 (in 9.999532 seconds) chunkcleanupworker stdout | 2023-08-08 06:40:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:12 UTC)" (scheduled at 2023-08-08 06:40:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:40:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:40:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 2, 294562), True, datetime.datetime(2023, 8, 8, 6, 40, 2, 294562), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:40:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:40:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:40:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:40:02,547 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:40:02,678 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:40:03,013 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:40:03,613 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:40:03,914 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:40:03,937 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:40:03,941 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:40:03,944 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:40:03,947 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:40:03,950 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:40:03,955 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:40:03,959 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:40:03,964 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:40:03,967 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:40:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:40:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:09.793658+00:00 (in 5.605699 seconds) repositorygcworker stdout | 2023-08-08 06:40:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:04 UTC)" (scheduled at 2023-08-08 06:40:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:40:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:40:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:04 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:40:04,410 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:40:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:40:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:15.371682+00:00 (in 9.999502 seconds) notificationworker stdout | 2023-08-08 06:40:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:15 UTC)" (scheduled at 2023-08-08 06:40:05.371682+00:00) notificationworker stdout | 2023-08-08 06:40:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:40:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 5, 372545), True, datetime.datetime(2023, 8, 8, 6, 40, 5, 372545), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:40:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:40:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:40:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:15 UTC)" executed successfully builder stdout | 2023-08-08 06:40:06,422 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:40:06,432 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:40:06,432 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:40:06,436 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:07,179 [197] [DEBUG] [app] Starting request: urn:request:9ade4801-acc9-4bb9-852d-ae0d42195f10 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:40:07,179 [200] [DEBUG] [app] Starting request: urn:request:f47f3f90-7a0e-4481-ba51-8e6b2806a8b3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:40:07,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:40:07,184 [203] [DEBUG] [app] Starting request: urn:request:891407e3-cc1c-4337-b66e-afe2db264287 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:40:07,185 [203] [DEBUG] [app] Ending request: urn:request:891407e3-cc1c-4337-b66e-afe2db264287 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:891407e3-cc1c-4337-b66e-afe2db264287', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:40:07,185 [210] [DEBUG] [app] Starting request: urn:request:b9fc8fdb-850c-4363-89f2-35533012c4b3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:40:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:40:07,186 [210] [DEBUG] [app] Ending request: urn:request:b9fc8fdb-850c-4363-89f2-35533012c4b3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b9fc8fdb-850c-4363-89f2-35533012c4b3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:07,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:40:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:40:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:07,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:07,191 [197] [DEBUG] [app] Starting request: urn:request:c0c7f2cf-5b7d-48b6-9b78-6fce94c892e1 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:40:07,191 [198] [DEBUG] [app] Starting request: urn:request:751228a2-8480-4608-a8c4-7f27ec954b67 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:40:07,191 [197] [DEBUG] [app] Ending request: urn:request:c0c7f2cf-5b7d-48b6-9b78-6fce94c892e1 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c0c7f2cf-5b7d-48b6-9b78-6fce94c892e1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:07,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:40:07,192 [198] [DEBUG] [app] Ending request: urn:request:751228a2-8480-4608-a8c4-7f27ec954b67 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:751228a2-8480-4608-a8c4-7f27ec954b67', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:07,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:07,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:40:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:40:07,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:40:07,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:40:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:40:07,198 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:40:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:37.197412+00:00 (in 29.999286 seconds) buildlogsarchiver stdout | 2023-08-08 06:40:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:37 UTC)" (scheduled at 2023-08-08 06:40:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:40:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 40, 7, 198603), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:40:07,202 [200] [DEBUG] [app] Ending request: urn:request:f47f3f90-7a0e-4481-ba51-8e6b2806a8b3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f47f3f90-7a0e-4481-ba51-8e6b2806a8b3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:40:07,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:07,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:40:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:40:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:40:07,203 [197] [DEBUG] [app] Ending request: urn:request:9ade4801-acc9-4bb9-852d-ae0d42195f10 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9ade4801-acc9-4bb9-852d-ae0d42195f10', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:40:07,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:40:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:40:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:40:07,207 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:40:07,208 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:40:07,209 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:40:07,212 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:40:07,212 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:40:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:40:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:19.417660+00:00 (in 9.999485 seconds) gcworker stdout | 2023-08-08 06:40:09,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:19 UTC)" (scheduled at 2023-08-08 06:40:09.417660+00:00) gcworker stdout | 2023-08-08 06:40:09,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267209419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:40:09,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:40:09,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:40:09,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:40:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:40:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:04.187429+00:00 (in 54.393223 seconds) repositorygcworker stdout | 2023-08-08 06:40:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:09 UTC)" (scheduled at 2023-08-08 06:40:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:40:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:40:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 9, 794583), True, datetime.datetime(2023, 8, 8, 6, 40, 9, 794583), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:40:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:40:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:40:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:40:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:40:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:22.293757+00:00 (in 9.999481 seconds) chunkcleanupworker stdout | 2023-08-08 06:40:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:22 UTC)" (scheduled at 2023-08-08 06:40:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:40:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:40:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 12, 294684), True, datetime.datetime(2023, 8, 8, 6, 40, 12, 294684), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:40:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:40:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:40:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:40:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:40:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:54.284093+00:00 (in 41.186236 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:40:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:13 UTC)" (scheduled at 2023-08-08 06:40:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:40:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:40:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 13, 98330), True, datetime.datetime(2023, 8, 8, 6, 40, 13, 98330), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:40:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:40:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:40:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:40:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:40:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:25.371682+00:00 (in 9.999511 seconds) notificationworker stdout | 2023-08-08 06:40:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:25 UTC)" (scheduled at 2023-08-08 06:40:15.371682+00:00) notificationworker stdout | 2023-08-08 06:40:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:40:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 15, 372646), True, datetime.datetime(2023, 8, 8, 6, 40, 15, 372646), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:40:15,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:40:15,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:40:15,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:25 UTC)" executed successfully builder stdout | 2023-08-08 06:40:16,446 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:40:16,455 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:40:16,455 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:40:16,459 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:40:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:40:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:29.417660+00:00 (in 9.999491 seconds) gcworker stdout | 2023-08-08 06:40:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:29 UTC)" (scheduled at 2023-08-08 06:40:19.417660+00:00) gcworker stdout | 2023-08-08 06:40:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267219418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:40:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:40:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:40:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:40:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:40:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:26.890741+00:00 (in 7.202907 seconds) securityworker stdout | 2023-08-08 06:40:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:49 UTC)" (scheduled at 2023-08-08 06:40:19.687363+00:00) securityworker stdout | 2023-08-08 06:40:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:40:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:40:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:40:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:40:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:40:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:40:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 35, 19, 691974), 1, 2]) securityworker stdout | 2023-08-08 06:40:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 35, 19, 691974), 1, 2]) securityworker stdout | 2023-08-08 06:40:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:40:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:49 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:40:19,831 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:40:20,857 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:40:22,040 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:40:22,179 [200] [DEBUG] [app] Starting request: urn:request:e7d76b93-04e0-4b1f-8f79-3e60c5932970 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:40:22,179 [199] [DEBUG] [app] Starting request: urn:request:ceacbc44-d776-456e-aba8-9aa780622838 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:40:22,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:40:22,184 [208] [DEBUG] [app] Starting request: urn:request:0f951d46-6172-4eae-9f3e-03bb9bb8c379 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:40:22,184 [207] [DEBUG] [app] Starting request: urn:request:f60e2d9d-5a49-4450-8974-1d277ecf6bdc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:40:22,185 [207] [DEBUG] [app] Ending request: urn:request:f60e2d9d-5a49-4450-8974-1d277ecf6bdc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f60e2d9d-5a49-4450-8974-1d277ecf6bdc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:40:22,185 [208] [DEBUG] [app] Ending request: urn:request:0f951d46-6172-4eae-9f3e-03bb9bb8c379 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0f951d46-6172-4eae-9f3e-03bb9bb8c379', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:40:22,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:40:22,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:40:22,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:22,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:22,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:22,190 [200] [DEBUG] [app] Starting request: urn:request:65dd910f-b0b2-4254-9d45-4d18b817ddf8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:40:22,191 [200] [DEBUG] [app] Ending request: urn:request:65dd910f-b0b2-4254-9d45-4d18b817ddf8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:65dd910f-b0b2-4254-9d45-4d18b817ddf8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:22,191 [198] [DEBUG] [app] Starting request: urn:request:4e80585b-f288-45c8-b2cc-64921c2860a6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:40:22,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:40:22,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:22,192 [198] [DEBUG] [app] Ending request: urn:request:4e80585b-f288-45c8-b2cc-64921c2860a6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4e80585b-f288-45c8-b2cc-64921c2860a6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:22,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:40:22,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:40:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:40:22,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:40:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:40:22,201 [200] [DEBUG] [app] Ending request: urn:request:e7d76b93-04e0-4b1f-8f79-3e60c5932970 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e7d76b93-04e0-4b1f-8f79-3e60c5932970', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:40:22,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:22,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:40:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:40:22,202 [199] [DEBUG] [app] Ending request: urn:request:ceacbc44-d776-456e-aba8-9aa780622838 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ceacbc44-d776-456e-aba8-9aa780622838', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:40:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:40:22,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:22,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:40:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:40:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:40:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:40:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:28.874906+00:00 (in 6.580652 seconds) chunkcleanupworker stdout | 2023-08-08 06:40:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:32 UTC)" (scheduled at 2023-08-08 06:40:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:40:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:40:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 22, 294749), True, datetime.datetime(2023, 8, 8, 6, 40, 22, 294749), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:40:22,301 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:40:22,301 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:40:22,301 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:32 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:40:22,505 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gcworker stdout | 2023-08-08 06:40:25,186 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:40:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:40:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:35.371682+00:00 (in 9.999507 seconds) notificationworker stdout | 2023-08-08 06:40:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:35 UTC)" (scheduled at 2023-08-08 06:40:25.371682+00:00) notificationworker stdout | 2023-08-08 06:40:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:40:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 25, 372676), True, datetime.datetime(2023, 8, 8, 6, 40, 25, 372676), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:40:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:40:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:40:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:40:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:40:25,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:47.398551+00:00 (in 21.883718 seconds) exportactionlogsworker stdout | 2023-08-08 06:40:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:25 UTC)" (scheduled at 2023-08-08 06:40:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:40:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:40:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:25 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:40:25,579 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:40:25,582 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:40:25,585 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:40:26,463 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:40:26,472 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:40:26,472 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:40:26,476 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:40:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:40:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:49.687363+00:00 (in 22.796125 seconds) securityworker stdout | 2023-08-08 06:40:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:56 UTC)" (scheduled at 2023-08-08 06:40:26.890741+00:00) securityworker stdout | 2023-08-08 06:40:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:40:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:40:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:40:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:40:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:40:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:40:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:40:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:40:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:40:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:40:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 35, 26, 897578), 1, 2]) securityworker stdout | 2023-08-08 06:40:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:26,909 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 35, 26, 897578), 1, 2]) securityworker stdout | 2023-08-08 06:40:26,911 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:26,911 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:26,911 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:40:26,911 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:40:26,911 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:40:26,911 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:40:26,911 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:40:26,911 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:40:26,912 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:40:26,912 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:40:56 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:40:27,063 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:40:27,419 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:40:27,451 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:40:27,971 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:40:28,281 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:40:28,407 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:40:28,421 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} gunicorn-web stdout | 2023-08-08 06:40:28,425 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} teamsyncworker stdout | 2023-08-08 06:40:28,482 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} chunkcleanupworker stdout | 2023-08-08 06:40:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:40:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:32.293757+00:00 (in 3.418343 seconds) chunkcleanupworker stdout | 2023-08-08 06:40:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:28 UTC)" (scheduled at 2023-08-08 06:40:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:40:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:40:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:28 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 06:40:28,978 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:40:29,169 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:40:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:40:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:39.417660+00:00 (in 9.999438 seconds) gcworker stdout | 2023-08-08 06:40:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:39 UTC)" (scheduled at 2023-08-08 06:40:29.417660+00:00) gcworker stdout | 2023-08-08 06:40:29,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267229418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:40:29,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:40:29,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:40:29,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:40:30,554 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:40:30,704 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:40:31,737 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:40:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:40:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:42.293757+00:00 (in 9.999431 seconds) chunkcleanupworker stdout | 2023-08-08 06:40:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:42 UTC)" (scheduled at 2023-08-08 06:40:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:40:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:40:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 32, 294872), True, datetime.datetime(2023, 8, 8, 6, 40, 32, 294872), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:40:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:40:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:40:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:40:32,571 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:40:32,718 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:40:33,021 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:40:33,651 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} namespacegcworker stdout | 2023-08-08 06:40:33,948 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:40:33,952 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:40:33,955 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:40:33,957 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:40:33,963 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:40:33,966 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:40:33,969 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:40:33,972 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:40:33,975 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:40:33,978 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} expiredappspecifictokenworker stdout | 2023-08-08 06:40:34,419 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:40:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:40:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:45.371682+00:00 (in 9.999527 seconds) notificationworker stdout | 2023-08-08 06:40:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:45 UTC)" (scheduled at 2023-08-08 06:40:35.371682+00:00) notificationworker stdout | 2023-08-08 06:40:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:40:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 35, 372723), True, datetime.datetime(2023, 8, 8, 6, 40, 35, 372723), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:40:35,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:40:35,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:40:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:45 UTC)" executed successfully builder stdout | 2023-08-08 06:40:36,478 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:40:36,487 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:40:36,487 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:40:36,491 [55] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:40:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:40:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:54.393627+00:00 (in 17.813282 seconds) namespacegcworker stdout | 2023-08-08 06:40:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:36 UTC)" (scheduled at 2023-08-08 06:40:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:40:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:40:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:36 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:40:37,179 [197] [DEBUG] [app] Starting request: urn:request:a984a7ab-567a-4306-a354-e12bb15c04b8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:40:37,180 [200] [DEBUG] [app] Starting request: urn:request:501a65a8-3e13-4282-89f9-0f69b3c5dbbe (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:40:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:40:37,184 [203] [DEBUG] [app] Starting request: urn:request:8821ea46-e168-4de6-8782-4a8de41307a0 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:40:37,184 [203] [DEBUG] [app] Ending request: urn:request:8821ea46-e168-4de6-8782-4a8de41307a0 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8821ea46-e168-4de6-8782-4a8de41307a0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:40:37,184 [206] [DEBUG] [app] Starting request: urn:request:2554a6c2-112b-4173-8e57-a4bce0a0339d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:40:37,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:40:37,185 [206] [DEBUG] [app] Ending request: urn:request:2554a6c2-112b-4173-8e57-a4bce0a0339d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2554a6c2-112b-4173-8e57-a4bce0a0339d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:40:37,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:40:37,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:37,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:37,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:37,190 [199] [DEBUG] [app] Starting request: urn:request:b39b49d0-8a06-4f7f-b787-0ef1e32fbead (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:40:37,190 [198] [DEBUG] [app] Starting request: urn:request:1c8b0d42-c162-4f25-8a9f-108ba1155c42 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:40:37,190 [199] [DEBUG] [app] Ending request: urn:request:b39b49d0-8a06-4f7f-b787-0ef1e32fbead (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b39b49d0-8a06-4f7f-b787-0ef1e32fbead', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:37,190 [198] [DEBUG] [app] Ending request: urn:request:1c8b0d42-c162-4f25-8a9f-108ba1155c42 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1c8b0d42-c162-4f25-8a9f-108ba1155c42', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:37,190 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:40:37,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:37,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:40:37,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:37,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:40:37,191 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:40:37,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:40:37,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:40:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:40:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:07.197412+00:00 (in 29.999547 seconds) buildlogsarchiver stdout | 2023-08-08 06:40:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:07 UTC)" (scheduled at 2023-08-08 06:40:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:40:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 40, 37, 198294), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:40:37,201 [200] [DEBUG] [app] Ending request: urn:request:501a65a8-3e13-4282-89f9-0f69b3c5dbbe (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:501a65a8-3e13-4282-89f9-0f69b3c5dbbe', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:40:37,201 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:37,201 [197] [DEBUG] [app] Ending request: urn:request:a984a7ab-567a-4306-a354-e12bb15c04b8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a984a7ab-567a-4306-a354-e12bb15c04b8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:40:37,201 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:37,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:40:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:40:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.022 47 0.023) gunicorn-web stdout | 2023-08-08 06:40:37,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:40:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:40:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:40:37,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:40:37,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:40:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:40:37,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:40:37,209 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:40:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:40:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:49.417660+00:00 (in 9.999385 seconds) gcworker stdout | 2023-08-08 06:40:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:49 UTC)" (scheduled at 2023-08-08 06:40:39.417660+00:00) gcworker stdout | 2023-08-08 06:40:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267239418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:40:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:40:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:40:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:40:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:40:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:52.293757+00:00 (in 9.999472 seconds) chunkcleanupworker stdout | 2023-08-08 06:40:42,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:52 UTC)" (scheduled at 2023-08-08 06:40:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:40:42,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:40:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 42, 295316), True, datetime.datetime(2023, 8, 8, 6, 40, 42, 295316), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:40:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:40:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:40:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:40:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:40:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:55.371682+00:00 (in 9.999467 seconds) notificationworker stdout | 2023-08-08 06:40:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:55 UTC)" (scheduled at 2023-08-08 06:40:45.371682+00:00) notificationworker stdout | 2023-08-08 06:40:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:40:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 45, 372703), True, datetime.datetime(2023, 8, 8, 6, 40, 45, 372703), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:40:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:40:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:40:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:55 UTC)" executed successfully builder stdout | 2023-08-08 06:40:46,499 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:40:46,508 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:40:46,508 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:40:46,513 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:40:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:40:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:25.514341+00:00 (in 38.115318 seconds) exportactionlogsworker stdout | 2023-08-08 06:40:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:47 UTC)" (scheduled at 2023-08-08 06:40:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:40:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:40:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 47, 399553), True, datetime.datetime(2023, 8, 8, 6, 40, 47, 399553), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:40:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:40:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:40:47,408 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:40:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:40:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:59.417660+00:00 (in 9.999507 seconds) gcworker stdout | 2023-08-08 06:40:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:59 UTC)" (scheduled at 2023-08-08 06:40:49.417660+00:00) gcworker stdout | 2023-08-08 06:40:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267249418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:40:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:40:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:40:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:40:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:40:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:40:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:40:56.890741+00:00 (in 7.202865 seconds) securityworker stdout | 2023-08-08 06:40:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:19 UTC)" (scheduled at 2023-08-08 06:40:49.687363+00:00) securityworker stdout | 2023-08-08 06:40:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:40:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:40:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:40:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:40:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:40:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:40:49,869 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:40:50,878 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:40:52,052 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:40:52,179 [197] [DEBUG] [app] Starting request: urn:request:24497ea7-d87b-421f-aaad-f9d42e5fc1ff (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:40:52,180 [198] [DEBUG] [app] Starting request: urn:request:f392874e-710a-467b-9deb-3f4c693cb76d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:40:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:52,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:52,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:40:52,185 [203] [DEBUG] [app] Starting request: urn:request:5e54cfe2-61a1-495f-b026-bfd027e1b635 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:40:52,185 [203] [DEBUG] [app] Ending request: urn:request:5e54cfe2-61a1-495f-b026-bfd027e1b635 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5e54cfe2-61a1-495f-b026-bfd027e1b635', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:40:52,186 [209] [DEBUG] [app] Starting request: urn:request:e158e8b3-0033-437f-bf6d-2f381678ff49 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:40:52,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:40:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:40:52,186 [209] [DEBUG] [app] Ending request: urn:request:e158e8b3-0033-437f-bf6d-2f381678ff49 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e158e8b3-0033-437f-bf6d-2f381678ff49', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:40:52,187 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:40:52,187 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:52,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:52,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:40:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:52,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:40:52,191 [200] [DEBUG] [app] Starting request: urn:request:56fa2bd9-570c-4629-863b-9724804e0366 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:40:52,191 [199] [DEBUG] [app] Starting request: urn:request:75216de0-3b33-4009-b477-64c209412eb2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:40:52,191 [200] [DEBUG] [app] Ending request: urn:request:56fa2bd9-570c-4629-863b-9724804e0366 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:56fa2bd9-570c-4629-863b-9724804e0366', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:40:52,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:40:52,192 [199] [DEBUG] [app] Ending request: urn:request:75216de0-3b33-4009-b477-64c209412eb2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:75216de0-3b33-4009-b477-64c209412eb2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:40:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:40:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:40:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:40:52,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:40:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:40:52,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:40:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:40:52,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:40:52,203 [198] [DEBUG] [app] Ending request: urn:request:f392874e-710a-467b-9deb-3f4c693cb76d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f392874e-710a-467b-9deb-3f4c693cb76d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:40:52,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:52,203 [197] [DEBUG] [app] Ending request: urn:request:24497ea7-d87b-421f-aaad-f9d42e5fc1ff (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:24497ea7-d87b-421f-aaad-f9d42e5fc1ff', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:40:52,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:40:52,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:40:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:40:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:40:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:40:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:40:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:40:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:40:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:02.293757+00:00 (in 9.999506 seconds) chunkcleanupworker stdout | 2023-08-08 06:40:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:02 UTC)" (scheduled at 2023-08-08 06:40:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:40:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:40:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 52, 294590), True, datetime.datetime(2023, 8, 8, 6, 40, 52, 294590), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:40:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:40:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:40:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:02 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:40:52,542 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:40:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:40:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:13.097375+00:00 (in 18.812781 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:40:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:54 UTC)" (scheduled at 2023-08-08 06:40:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:40:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:40:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:40:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:40:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:36.579841+00:00 (in 42.185693 seconds) namespacegcworker stdout | 2023-08-08 06:40:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:54 UTC)" (scheduled at 2023-08-08 06:40:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:40:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:40:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 54, 394589), True, datetime.datetime(2023, 8, 8, 6, 40, 54, 394589), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:40:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:40:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:40:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:41:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:40:55,223 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:40:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:40:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:02.178530+00:00 (in 6.806345 seconds) notificationworker stdout | 2023-08-08 06:40:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:05 UTC)" (scheduled at 2023-08-08 06:40:55.371682+00:00) notificationworker stdout | 2023-08-08 06:40:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:40:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 40, 55, 372654), True, datetime.datetime(2023, 8, 8, 6, 40, 55, 372654), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:40:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:40:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:40:55,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:40:55,588 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:40:55,592 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:40:55,595 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:40:56,514 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:40:56,523 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:40:56,523 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:40:56,528 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:40:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:40:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:19.687363+00:00 (in 22.796079 seconds) securityworker stdout | 2023-08-08 06:40:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:26 UTC)" (scheduled at 2023-08-08 06:40:56.890741+00:00) securityworker stdout | 2023-08-08 06:40:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:40:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:40:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:40:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:40:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:40:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:40:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:40:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:40:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:40:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:40:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 35, 56, 896699), 1, 2]) securityworker stdout | 2023-08-08 06:40:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:56,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 35, 56, 896699), 1, 2]) securityworker stdout | 2023-08-08 06:40:56,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:40:56,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:40:56,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:40:56,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:40:56,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:40:56,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:40:56,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:40:56,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:40:56,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:40:56,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:26 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:40:57,099 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:40:57,433 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:40:57,489 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:40:57,982 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:40:58,295 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:40:58,422 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:40:58,438 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:40:58,445 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:40:58,496 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} repositoryactioncounter stdout | 2023-08-08 06:40:59,003 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:40:59,195 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:40:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:40:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:09.417660+00:00 (in 9.999450 seconds) gcworker stdout | 2023-08-08 06:40:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:09 UTC)" (scheduled at 2023-08-08 06:40:59.417660+00:00) gcworker stdout | 2023-08-08 06:40:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267259418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:40:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:40:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:40:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:41:00,582 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:41:00,721 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:41:01,756 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:41:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:41:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:05.371682+00:00 (in 3.192630 seconds) notificationworker stdout | 2023-08-08 06:41:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:02 UTC)" (scheduled at 2023-08-08 06:41:02.178530+00:00) notificationworker stdout | 2023-08-08 06:41:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:41:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:41:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:41:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:12.293757+00:00 (in 9.999500 seconds) chunkcleanupworker stdout | 2023-08-08 06:41:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:12 UTC)" (scheduled at 2023-08-08 06:41:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:41:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:41:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 2, 294632), True, datetime.datetime(2023, 8, 8, 6, 41, 2, 294632), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:41:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:41:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:41:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:41:02,609 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:41:02,756 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:41:03,059 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:41:03,689 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:41:03,965 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:41:03,969 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:41:03,972 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:41:03,975 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:41:03,980 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:41:03,983 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:41:03,987 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} namespacegcworker stdout | 2023-08-08 06:41:03,990 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} gunicorn-registry stdout | 2023-08-08 06:41:03,993 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:41:03,997 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:41:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:41:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:09.793658+00:00 (in 5.605745 seconds) repositorygcworker stdout | 2023-08-08 06:41:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:04 UTC)" (scheduled at 2023-08-08 06:41:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:41:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:41:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:04 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:41:04,457 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:41:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:41:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:15.371682+00:00 (in 9.999467 seconds) notificationworker stdout | 2023-08-08 06:41:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:15 UTC)" (scheduled at 2023-08-08 06:41:05.371682+00:00) notificationworker stdout | 2023-08-08 06:41:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:41:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 5, 372823), True, datetime.datetime(2023, 8, 8, 6, 41, 5, 372823), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:41:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:41:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:41:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:15 UTC)" executed successfully builder stdout | 2023-08-08 06:41:06,538 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:41:06,547 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:41:06,548 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:41:06,552 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:41:07,178 [199] [DEBUG] [app] Starting request: urn:request:0ebc78c3-30e5-4cb4-b90a-ecf02d98bae7 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:41:07,179 [197] [DEBUG] [app] Starting request: urn:request:7a678297-7353-4a22-bbb6-7819dcf750fd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:41:07,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:07,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:07,182 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:41:07,183 [209] [DEBUG] [app] Starting request: urn:request:9a3f36ae-8a0d-4e00-aa6e-af8e5c1a73a2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:41:07,184 [209] [DEBUG] [app] Ending request: urn:request:9a3f36ae-8a0d-4e00-aa6e-af8e5c1a73a2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9a3f36ae-8a0d-4e00-aa6e-af8e5c1a73a2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:41:07,184 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:07,184 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:41:07,185 [209] [DEBUG] [app] Starting request: urn:request:ded081de-7c91-4086-80ed-f7a62cf52cc5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:41:07,185 [209] [DEBUG] [app] Ending request: urn:request:ded081de-7c91-4086-80ed-f7a62cf52cc5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ded081de-7c91-4086-80ed-f7a62cf52cc5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:41:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.001) gunicorn-web stdout | 2023-08-08 06:41:07,186 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:07,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:07,188 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:07,189 [199] [DEBUG] [app] Starting request: urn:request:9fac2672-17d0-4b64-be53-2137dba7cf7b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:07,189 [199] [DEBUG] [app] Ending request: urn:request:9fac2672-17d0-4b64-be53-2137dba7cf7b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9fac2672-17d0-4b64-be53-2137dba7cf7b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:41:07,189 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:41:07,190 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:07,190 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:41:07,191 [198] [DEBUG] [app] Starting request: urn:request:90da4001-834d-4c86-8744-f757e8179d14 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:07,191 [198] [DEBUG] [app] Ending request: urn:request:90da4001-834d-4c86-8744-f757e8179d14 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:90da4001-834d-4c86-8744-f757e8179d14', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:41:07,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:41:07,195 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:41:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:41:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:41:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:37.197412+00:00 (in 29.999512 seconds) buildlogsarchiver stdout | 2023-08-08 06:41:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:37 UTC)" (scheduled at 2023-08-08 06:41:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:41:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 41, 7, 198322), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:41:07,201 [199] [DEBUG] [app] Ending request: urn:request:0ebc78c3-30e5-4cb4-b90a-ecf02d98bae7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0ebc78c3-30e5-4cb4-b90a-ecf02d98bae7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:41:07,201 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:41:07,201 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:41:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:41:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:41:07,202 [197] [DEBUG] [app] Ending request: urn:request:7a678297-7353-4a22-bbb6-7819dcf750fd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7a678297-7353-4a22-bbb6-7819dcf750fd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:41:07,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:41:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:41:07,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:41:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:41:07,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:41:07,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:41:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:41:07,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:41:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:41:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:41:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:19.417660+00:00 (in 9.999475 seconds) gcworker stdout | 2023-08-08 06:41:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:19 UTC)" (scheduled at 2023-08-08 06:41:09.417660+00:00) gcworker stdout | 2023-08-08 06:41:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267269418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:41:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:41:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:41:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:41:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:41:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:04.187429+00:00 (in 54.393259 seconds) repositorygcworker stdout | 2023-08-08 06:41:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:09 UTC)" (scheduled at 2023-08-08 06:41:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:41:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:41:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 9, 794495), True, datetime.datetime(2023, 8, 8, 6, 41, 9, 794495), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:41:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:41:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:41:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:41:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:41:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:22.293757+00:00 (in 9.999482 seconds) chunkcleanupworker stdout | 2023-08-08 06:41:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:22 UTC)" (scheduled at 2023-08-08 06:41:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:41:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:41:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 12, 294846), True, datetime.datetime(2023, 8, 8, 6, 41, 12, 294846), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:41:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:41:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:41:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:41:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:41:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:31.283850+00:00 (in 18.185947 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:41:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:13 UTC)" (scheduled at 2023-08-08 06:41:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:41:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:41:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 13, 98377), True, datetime.datetime(2023, 8, 8, 6, 41, 13, 98377), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:41:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:41:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:41:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:41:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:41:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:25.371682+00:00 (in 9.999493 seconds) notificationworker stdout | 2023-08-08 06:41:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:25 UTC)" (scheduled at 2023-08-08 06:41:15.371682+00:00) notificationworker stdout | 2023-08-08 06:41:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:41:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 15, 372605), True, datetime.datetime(2023, 8, 8, 6, 41, 15, 372605), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:41:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:41:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:41:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:25 UTC)" executed successfully builder stdout | 2023-08-08 06:41:16,562 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:41:16,573 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:41:16,573 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:41:16,579 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:41:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:41:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:29.417660+00:00 (in 9.999444 seconds) gcworker stdout | 2023-08-08 06:41:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:29 UTC)" (scheduled at 2023-08-08 06:41:19.417660+00:00) gcworker stdout | 2023-08-08 06:41:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267279418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:41:19,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:41:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:41:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:41:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:41:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:26.890741+00:00 (in 7.202900 seconds) securityworker stdout | 2023-08-08 06:41:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:49 UTC)" (scheduled at 2023-08-08 06:41:19.687363+00:00) securityworker stdout | 2023-08-08 06:41:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:41:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:41:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:41:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:41:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:41:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:41:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 36, 19, 691878), 1, 2]) securityworker stdout | 2023-08-08 06:41:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 36, 19, 691878), 1, 2]) securityworker stdout | 2023-08-08 06:41:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:41:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:49 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:41:19,907 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:41:20,916 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:41:22,082 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:41:22,178 [197] [DEBUG] [app] Starting request: urn:request:ba731e25-d1f1-4441-9a03-7c0e4542e08c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:41:22,179 [200] [DEBUG] [app] Starting request: urn:request:0b794aca-49af-412e-8dc9-1115beeee7de (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:41:22,179 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:22,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:22,182 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:22,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:41:22,184 [209] [DEBUG] [app] Starting request: urn:request:968f94b3-45ab-4c73-b88b-669e2738bc3d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:41:22,184 [210] [DEBUG] [app] Starting request: urn:request:2866db06-c614-499b-9427-071c8c7d3554 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:41:22,184 [209] [DEBUG] [app] Ending request: urn:request:968f94b3-45ab-4c73-b88b-669e2738bc3d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:968f94b3-45ab-4c73-b88b-669e2738bc3d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:41:22,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:41:22,185 [210] [DEBUG] [app] Ending request: urn:request:2866db06-c614-499b-9427-071c8c7d3554 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2866db06-c614-499b-9427-071c8c7d3554', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:41:22,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:41:22,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:41:22,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:22,186 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:22,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:22,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:22,189 [200] [DEBUG] [app] Starting request: urn:request:ff8b783b-18cc-4191-9ba7-d29d3a8a923e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:22,190 [200] [DEBUG] [app] Ending request: urn:request:ff8b783b-18cc-4191-9ba7-d29d3a8a923e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ff8b783b-18cc-4191-9ba7-d29d3a8a923e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:41:22,190 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:22,190 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:22,191 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:22,191 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:41:22,192 [199] [DEBUG] [app] Starting request: urn:request:7439cebd-05ae-4d43-aee2-08dce2e9ee0a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:22,192 [199] [DEBUG] [app] Ending request: urn:request:7439cebd-05ae-4d43-aee2-08dce2e9ee0a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7439cebd-05ae-4d43-aee2-08dce2e9ee0a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:41:22,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:22,193 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:22,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:41:22,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:41:22,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:41:22,202 [197] [DEBUG] [app] Ending request: urn:request:ba731e25-d1f1-4441-9a03-7c0e4542e08c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ba731e25-d1f1-4441-9a03-7c0e4542e08c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:41:22,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:41:22,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:41:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:41:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:41:22,203 [200] [DEBUG] [app] Ending request: urn:request:0b794aca-49af-412e-8dc9-1115beeee7de (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0b794aca-49af-412e-8dc9-1115beeee7de', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:41:22,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:41:22,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:41:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:41:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:41:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:41:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:28.874906+00:00 (in 6.580627 seconds) chunkcleanupworker stdout | 2023-08-08 06:41:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:32 UTC)" (scheduled at 2023-08-08 06:41:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:41:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:41:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 22, 294794), True, datetime.datetime(2023, 8, 8, 6, 41, 22, 294794), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:41:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:41:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:41:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:32 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:41:22,577 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gcworker stdout | 2023-08-08 06:41:25,263 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:41:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:41:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:35.371682+00:00 (in 9.999564 seconds) notificationworker stdout | 2023-08-08 06:41:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:35 UTC)" (scheduled at 2023-08-08 06:41:25.371682+00:00) notificationworker stdout | 2023-08-08 06:41:25,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:41:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 25, 373081), True, datetime.datetime(2023, 8, 8, 6, 41, 25, 373081), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:41:25,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:41:25,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:41:25,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:41:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:41:25,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:47.398551+00:00 (in 21.883704 seconds) exportactionlogsworker stdout | 2023-08-08 06:41:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:25 UTC)" (scheduled at 2023-08-08 06:41:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:41:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:41:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:25 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:41:25,599 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:41:25,603 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:41:25,605 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:41:26,581 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:41:26,591 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:41:26,591 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:41:26,597 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:41:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:41:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:49.687363+00:00 (in 22.796116 seconds) securityworker stdout | 2023-08-08 06:41:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:56 UTC)" (scheduled at 2023-08-08 06:41:26.890741+00:00) securityworker stdout | 2023-08-08 06:41:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:41:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:41:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:41:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:41:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:41:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:41:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:41:26,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:41:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:41:26,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:41:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 36, 26, 896073), 1, 2]) securityworker stdout | 2023-08-08 06:41:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 36, 26, 896073), 1, 2]) securityworker stdout | 2023-08-08 06:41:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:41:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:41:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:41:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:41:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:41:26,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:41:26,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:41:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:41:56 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:41:27,108 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:41:27,440 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:41:27,527 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:41:27,990 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:41:28,308 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:41:28,436 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:41:28,450 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:41:28,457 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:41:28,533 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} chunkcleanupworker stdout | 2023-08-08 06:41:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:41:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:32.293757+00:00 (in 3.418338 seconds) chunkcleanupworker stdout | 2023-08-08 06:41:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:28 UTC)" (scheduled at 2023-08-08 06:41:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:41:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:41:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:28 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 06:41:29,013 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:41:29,211 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:41:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:41:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:39.417660+00:00 (in 9.999437 seconds) gcworker stdout | 2023-08-08 06:41:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:39 UTC)" (scheduled at 2023-08-08 06:41:29.417660+00:00) gcworker stdout | 2023-08-08 06:41:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267289418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:41:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:41:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:41:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:41:30,601 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:41:30,742 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityscanningnotificationworker stdout | 2023-08-08 06:41:31,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:41:31,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:54.284093+00:00 (in 22.999715 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:41:31,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:46:31 UTC)" (scheduled at 2023-08-08 06:41:31.283850+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:41:31,285 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 41, 31, 284886), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:41:31,293 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 41, 31, 284886), True, datetime.datetime(2023, 8, 8, 6, 41, 31, 284886), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:41:31,295 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 41, 31, 284886), True, datetime.datetime(2023, 8, 8, 6, 41, 31, 284886), 0, 'secscanv4/%', False, datetime.datetime(2023, 8, 8, 6, 41, 31, 284886), 'secscanv4/%']) securityscanningnotificationworker stdout | 2023-08-08 06:41:31,296 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:41:31,297 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:46:31 UTC)" executed successfully securityworker stdout | 2023-08-08 06:41:31,794 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:41:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:41:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:42.293757+00:00 (in 9.999329 seconds) chunkcleanupworker stdout | 2023-08-08 06:41:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:42 UTC)" (scheduled at 2023-08-08 06:41:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:41:32,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:41:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 32, 295047), True, datetime.datetime(2023, 8, 8, 6, 41, 32, 295047), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:41:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:41:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:41:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:41:32,632 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:41:32,796 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:41:33,096 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:41:33,730 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:41:33,978 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:41:33,981 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:41:33,984 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:41:33,990 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:41:33,993 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:41:33,996 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:41:33,999 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:41:34,002 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:41:34,010 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} namespacegcworker stdout | 2023-08-08 06:41:34,025 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:41:34,494 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:41:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:41:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:45.371682+00:00 (in 9.999555 seconds) notificationworker stdout | 2023-08-08 06:41:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:45 UTC)" (scheduled at 2023-08-08 06:41:35.371682+00:00) notificationworker stdout | 2023-08-08 06:41:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:41:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 35, 372460), True, datetime.datetime(2023, 8, 8, 6, 41, 35, 372460), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:41:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:41:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:41:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:41:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:41:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:54.393627+00:00 (in 17.813207 seconds) namespacegcworker stdout | 2023-08-08 06:41:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:36 UTC)" (scheduled at 2023-08-08 06:41:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:41:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:41:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:36 UTC)" executed successfully builder stdout | 2023-08-08 06:41:36,598 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:41:36,607 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:41:36,607 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:41:36,612 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:41:37,178 [197] [DEBUG] [app] Starting request: urn:request:9196c2cc-d8c3-42c1-92d4-bc821baed395 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:41:37,179 [198] [DEBUG] [app] Starting request: urn:request:8833e21a-a0e9-4cdc-8046-ec1587b40907 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:41:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:37,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:37,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:41:37,184 [206] [DEBUG] [app] Starting request: urn:request:634a63ea-34bb-4fcd-85bf-ba52d9f4111a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:37,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:41:37,184 [206] [DEBUG] [app] Ending request: urn:request:634a63ea-34bb-4fcd-85bf-ba52d9f4111a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:634a63ea-34bb-4fcd-85bf-ba52d9f4111a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:41:37,185 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:37,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:41:37,185 [203] [DEBUG] [app] Starting request: urn:request:b02bca5b-0d90-4dea-83aa-2b13f277a95e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:41:37,186 [203] [DEBUG] [app] Ending request: urn:request:b02bca5b-0d90-4dea-83aa-2b13f277a95e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b02bca5b-0d90-4dea-83aa-2b13f277a95e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:41:37,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:37,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:37,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:37,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:37,191 [199] [DEBUG] [app] Starting request: urn:request:4f771123-eb00-431e-9772-9d9d113ffbd4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:37,191 [200] [DEBUG] [app] Starting request: urn:request:7c699c94-4902-4e6c-84da-81efaff0813a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:37,191 [200] [DEBUG] [app] Ending request: urn:request:7c699c94-4902-4e6c-84da-81efaff0813a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7c699c94-4902-4e6c-84da-81efaff0813a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:41:37,191 [199] [DEBUG] [app] Ending request: urn:request:4f771123-eb00-431e-9772-9d9d113ffbd4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4f771123-eb00-431e-9772-9d9d113ffbd4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:41:37,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:37,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:41:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:37,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:41:37,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:41:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:41:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:41:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:07.197412+00:00 (in 29.999306 seconds) buildlogsarchiver stdout | 2023-08-08 06:41:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:07 UTC)" (scheduled at 2023-08-08 06:41:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:41:37,198 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:41:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 41, 37, 198600), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:41:37,203 [197] [DEBUG] [app] Ending request: urn:request:9196c2cc-d8c3-42c1-92d4-bc821baed395 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9196c2cc-d8c3-42c1-92d4-bc821baed395', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:41:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:41:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:41:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:41:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:41:37,204 [198] [DEBUG] [app] Ending request: urn:request:8833e21a-a0e9-4cdc-8046-ec1587b40907 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8833e21a-a0e9-4cdc-8046-ec1587b40907', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:41:37,204 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:41:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.027) gunicorn-web stdout | 2023-08-08 06:41:37,204 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:41:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:41:37,205 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:41:37,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:41:37,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:41:37,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:41:37,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:41:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:41:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:49.417660+00:00 (in 9.999454 seconds) gcworker stdout | 2023-08-08 06:41:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:49 UTC)" (scheduled at 2023-08-08 06:41:39.417660+00:00) gcworker stdout | 2023-08-08 06:41:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267299418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:41:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:41:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:41:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:41:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:41:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:52.293757+00:00 (in 9.999309 seconds) chunkcleanupworker stdout | 2023-08-08 06:41:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:52 UTC)" (scheduled at 2023-08-08 06:41:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:41:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:41:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 42, 295019), True, datetime.datetime(2023, 8, 8, 6, 41, 42, 295019), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:41:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:41:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:41:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:41:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:41:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:55.371682+00:00 (in 9.999477 seconds) notificationworker stdout | 2023-08-08 06:41:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:55 UTC)" (scheduled at 2023-08-08 06:41:45.371682+00:00) notificationworker stdout | 2023-08-08 06:41:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:41:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 45, 372561), True, datetime.datetime(2023, 8, 8, 6, 41, 45, 372561), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:41:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:41:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:41:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:55 UTC)" executed successfully builder stdout | 2023-08-08 06:41:46,622 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:41:46,631 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:41:46,631 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:41:46,635 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:41:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:41:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:25.514341+00:00 (in 38.115359 seconds) exportactionlogsworker stdout | 2023-08-08 06:41:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:47 UTC)" (scheduled at 2023-08-08 06:41:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:41:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:41:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 47, 399332), True, datetime.datetime(2023, 8, 8, 6, 41, 47, 399332), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:41:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:41:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:41:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:41:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:41:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:59.417660+00:00 (in 9.999465 seconds) gcworker stdout | 2023-08-08 06:41:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:59 UTC)" (scheduled at 2023-08-08 06:41:49.417660+00:00) gcworker stdout | 2023-08-08 06:41:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267309418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:41:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:41:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:41:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:41:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:41:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:41:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:41:56.890741+00:00 (in 7.202802 seconds) securityworker stdout | 2023-08-08 06:41:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:19 UTC)" (scheduled at 2023-08-08 06:41:49.687363+00:00) securityworker stdout | 2023-08-08 06:41:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:41:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:41:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:41:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:41:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:41:49,699 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:41:49,944 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:41:50,930 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:41:52,096 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:41:52,178 [199] [DEBUG] [app] Starting request: urn:request:23b7f39d-b90a-4937-80b0-477430cce318 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:41:52,179 [200] [DEBUG] [app] Starting request: urn:request:1375693e-3e44-41a2-b613-895596c928b4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:41:52,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:52,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:41:52,184 [203] [DEBUG] [app] Starting request: urn:request:5a2ce4ab-4ca2-4540-bdc7-3ca122588572 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:41:52,185 [203] [DEBUG] [app] Ending request: urn:request:5a2ce4ab-4ca2-4540-bdc7-3ca122588572 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5a2ce4ab-4ca2-4540-bdc7-3ca122588572', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:41:52,185 [210] [DEBUG] [app] Starting request: urn:request:2644617b-99fa-4059-ae15-b060a7114045 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:41:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:41:52,185 [210] [DEBUG] [app] Ending request: urn:request:2644617b-99fa-4059-ae15-b060a7114045 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2644617b-99fa-4059-ae15-b060a7114045', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:41:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:41:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:52,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:41:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:41:52,190 [197] [DEBUG] [app] Starting request: urn:request:3aa2d1a9-b497-4eb6-9660-e3bc76065a09 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:52,191 [199] [DEBUG] [app] Starting request: urn:request:272e5a15-4982-4442-bb7a-5bd2211da64d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:41:52,191 [197] [DEBUG] [app] Ending request: urn:request:3aa2d1a9-b497-4eb6-9660-e3bc76065a09 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3aa2d1a9-b497-4eb6-9660-e3bc76065a09', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:41:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:41:52,191 [199] [DEBUG] [app] Ending request: urn:request:272e5a15-4982-4442-bb7a-5bd2211da64d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:272e5a15-4982-4442-bb7a-5bd2211da64d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:41:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:52,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:41:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:41:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:41:52,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:41:52,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:41:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:41:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:41:52,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:41:52,202 [200] [DEBUG] [app] Ending request: urn:request:1375693e-3e44-41a2-b613-895596c928b4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1375693e-3e44-41a2-b613-895596c928b4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:41:52,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:41:52,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:41:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:41:52,203 [199] [DEBUG] [app] Ending request: urn:request:23b7f39d-b90a-4937-80b0-477430cce318 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:23b7f39d-b90a-4937-80b0-477430cce318', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:41:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) gunicorn-web stdout | 2023-08-08 06:41:52,203 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:41:52,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:41:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:41:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:41:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:41:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:02.293757+00:00 (in 9.999414 seconds) chunkcleanupworker stdout | 2023-08-08 06:41:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:02 UTC)" (scheduled at 2023-08-08 06:41:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:41:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:41:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 52, 294843), True, datetime.datetime(2023, 8, 8, 6, 41, 52, 294843), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:41:52,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:41:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:41:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:02 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:41:52,602 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:41:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:41:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:13.097375+00:00 (in 18.812730 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:41:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:54 UTC)" (scheduled at 2023-08-08 06:41:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:41:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:41:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:41:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:41:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:36.579841+00:00 (in 42.185684 seconds) namespacegcworker stdout | 2023-08-08 06:41:54,395 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:54 UTC)" (scheduled at 2023-08-08 06:41:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:41:54,395 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:41:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 54, 395213), True, datetime.datetime(2023, 8, 8, 6, 41, 54, 395213), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:41:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:41:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:41:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:42:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:41:55,300 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:41:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:41:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:02.178530+00:00 (in 6.806361 seconds) notificationworker stdout | 2023-08-08 06:41:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:05 UTC)" (scheduled at 2023-08-08 06:41:55.371682+00:00) notificationworker stdout | 2023-08-08 06:41:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:41:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 41, 55, 372507), True, datetime.datetime(2023, 8, 8, 6, 41, 55, 372507), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:41:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:41:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:41:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:41:55,610 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:41:55,613 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:41:55,616 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:41:56,646 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:41:56,654 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:41:56,654 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:41:56,658 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:41:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:41:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:19.687363+00:00 (in 22.795976 seconds) securityworker stdout | 2023-08-08 06:41:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:26 UTC)" (scheduled at 2023-08-08 06:41:56.890741+00:00) securityworker stdout | 2023-08-08 06:41:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:41:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:41:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:41:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:41:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:41:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:41:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:41:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:41:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:41:56,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:41:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 36, 56, 896867), 1, 2]) securityworker stdout | 2023-08-08 06:41:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 36, 56, 896867), 1, 2]) securityworker stdout | 2023-08-08 06:41:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:41:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:41:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:41:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:41:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:41:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:41:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:41:56,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:41:56,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:41:56,908 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:26 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:41:57,145 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:41:57,478 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:41:57,545 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:41:58,000 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:41:58,320 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:41:58,450 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:41:58,463 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:41:58,471 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:41:58,541 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} repositoryactioncounter stdout | 2023-08-08 06:41:59,020 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:41:59,248 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:41:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:41:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:09.417660+00:00 (in 9.999445 seconds) gcworker stdout | 2023-08-08 06:41:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:09 UTC)" (scheduled at 2023-08-08 06:41:59.417660+00:00) gcworker stdout | 2023-08-08 06:41:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267319418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:41:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:41:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:41:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:42:00,634 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:42:00,780 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:42:01,827 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:42:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:42:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:05.371682+00:00 (in 3.192677 seconds) notificationworker stdout | 2023-08-08 06:42:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:02 UTC)" (scheduled at 2023-08-08 06:42:02.178530+00:00) notificationworker stdout | 2023-08-08 06:42:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:42:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:42:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:42:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:12.293757+00:00 (in 9.999441 seconds) chunkcleanupworker stdout | 2023-08-08 06:42:02,295 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:12 UTC)" (scheduled at 2023-08-08 06:42:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:42:02,295 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:42:02,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 2, 295315), True, datetime.datetime(2023, 8, 8, 6, 42, 2, 295315), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:42:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:42:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:42:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:42:02,667 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:42:02,816 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:42:03,135 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:42:03,768 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:42:03,990 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:42:03,994 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:42:03,998 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:42:04,000 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:42:04,003 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:42:04,008 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:42:04,011 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:42:04,015 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:42:04,019 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} namespacegcworker stdout | 2023-08-08 06:42:04,057 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} repositorygcworker stdout | 2023-08-08 06:42:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:42:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:09.793658+00:00 (in 5.605736 seconds) repositorygcworker stdout | 2023-08-08 06:42:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:04 UTC)" (scheduled at 2023-08-08 06:42:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:42:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:42:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:04 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:42:04,532 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:42:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:42:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:15.371682+00:00 (in 9.999486 seconds) notificationworker stdout | 2023-08-08 06:42:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:15 UTC)" (scheduled at 2023-08-08 06:42:05.371682+00:00) notificationworker stdout | 2023-08-08 06:42:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:42:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 5, 372541), True, datetime.datetime(2023, 8, 8, 6, 42, 5, 372541), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:42:05,381 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:42:05,381 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:42:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:15 UTC)" executed successfully builder stdout | 2023-08-08 06:42:06,669 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:42:06,677 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:42:06,678 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:42:06,683 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:42:07,179 [197] [DEBUG] [app] Starting request: urn:request:519d5fd4-3af1-4cc0-aaba-64f657aecc29 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:42:07,179 [198] [DEBUG] [app] Starting request: urn:request:69150d69-f194-4b44-9891-ce601bcf82cb (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:42:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:07,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:07,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:42:07,185 [203] [DEBUG] [app] Starting request: urn:request:2088a63e-4d26-49b0-b5b5-96a5827fd1a6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:42:07,185 [204] [DEBUG] [app] Starting request: urn:request:17eb462b-d154-41e6-a640-272d5d96626a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:42:07,185 [203] [DEBUG] [app] Ending request: urn:request:2088a63e-4d26-49b0-b5b5-96a5827fd1a6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2088a63e-4d26-49b0-b5b5-96a5827fd1a6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:42:07,186 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:42:07,186 [204] [DEBUG] [app] Ending request: urn:request:17eb462b-d154-41e6-a640-272d5d96626a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:17eb462b-d154-41e6-a640-272d5d96626a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:42:07,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:42:07,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:07,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:07,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:07,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:07,190 [200] [DEBUG] [app] Starting request: urn:request:67ba2903-238b-4aeb-8ee9-0e6444061eb0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:07,191 [199] [DEBUG] [app] Starting request: urn:request:2d6f0540-34ab-472c-929c-1bfb8c697f22 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:07,191 [200] [DEBUG] [app] Ending request: urn:request:67ba2903-238b-4aeb-8ee9-0e6444061eb0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:67ba2903-238b-4aeb-8ee9-0e6444061eb0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:07,191 [199] [DEBUG] [app] Ending request: urn:request:2d6f0540-34ab-472c-929c-1bfb8c697f22 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2d6f0540-34ab-472c-929c-1bfb8c697f22', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:07,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:42:07,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:42:07,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:42:07,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:42:07,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:42:07,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:42:07,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:42:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:42:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:37.197412+00:00 (in 29.999506 seconds) buildlogsarchiver stdout | 2023-08-08 06:42:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:37 UTC)" (scheduled at 2023-08-08 06:42:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:42:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 42, 7, 198383), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:42:07,202 [198] [DEBUG] [app] Ending request: urn:request:69150d69-f194-4b44-9891-ce601bcf82cb (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:69150d69-f194-4b44-9891-ce601bcf82cb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:42:07,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:42:07,202 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:42:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:42:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.023) gunicorn-web stdout | 2023-08-08 06:42:07,203 [197] [DEBUG] [app] Ending request: urn:request:519d5fd4-3af1-4cc0-aaba-64f657aecc29 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:519d5fd4-3af1-4cc0-aaba-64f657aecc29', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:42:07,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:42:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:42:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:42:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:42:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:42:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:42:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:42:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:42:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:42:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:42:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:19.417660+00:00 (in 9.999424 seconds) gcworker stdout | 2023-08-08 06:42:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:19 UTC)" (scheduled at 2023-08-08 06:42:09.417660+00:00) gcworker stdout | 2023-08-08 06:42:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267329418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:42:09,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:42:09,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:42:09,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:42:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:42:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:04.187429+00:00 (in 54.393254 seconds) repositorygcworker stdout | 2023-08-08 06:42:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:09 UTC)" (scheduled at 2023-08-08 06:42:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:42:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:42:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 9, 794528), True, datetime.datetime(2023, 8, 8, 6, 42, 9, 794528), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:42:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:42:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:42:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:42:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:42:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:22.293757+00:00 (in 9.999456 seconds) chunkcleanupworker stdout | 2023-08-08 06:42:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:22 UTC)" (scheduled at 2023-08-08 06:42:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:42:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:42:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 12, 294702), True, datetime.datetime(2023, 8, 8, 6, 42, 12, 294702), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:42:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:42:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:42:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:42:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:42:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:54.284093+00:00 (in 41.186189 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:42:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:13 UTC)" (scheduled at 2023-08-08 06:42:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:42:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:42:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 13, 98367), True, datetime.datetime(2023, 8, 8, 6, 42, 13, 98367), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:42:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:42:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:42:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:42:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:42:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:25.371682+00:00 (in 9.999544 seconds) notificationworker stdout | 2023-08-08 06:42:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:25 UTC)" (scheduled at 2023-08-08 06:42:15.371682+00:00) notificationworker stdout | 2023-08-08 06:42:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:42:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 15, 372481), True, datetime.datetime(2023, 8, 8, 6, 42, 15, 372481), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:42:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:42:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:42:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:25 UTC)" executed successfully builder stdout | 2023-08-08 06:42:16,693 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:42:16,702 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:42:16,702 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:42:16,706 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:42:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:42:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:29.417660+00:00 (in 9.999515 seconds) gcworker stdout | 2023-08-08 06:42:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:29 UTC)" (scheduled at 2023-08-08 06:42:19.417660+00:00) gcworker stdout | 2023-08-08 06:42:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267339418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:42:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:42:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:42:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:42:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:42:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:26.890741+00:00 (in 7.202864 seconds) securityworker stdout | 2023-08-08 06:42:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:49 UTC)" (scheduled at 2023-08-08 06:42:19.687363+00:00) securityworker stdout | 2023-08-08 06:42:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:42:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:42:19,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:42:19,693 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:42:19,699 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:42:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:42:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:19,702 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 37, 19, 693036), 1, 2]) securityworker stdout | 2023-08-08 06:42:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:19,704 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 37, 19, 693036), 1, 2]) securityworker stdout | 2023-08-08 06:42:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:19,706 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:19,706 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:42:19,706 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:49 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:42:19,969 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:42:20,967 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:42:22,133 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:42:22,179 [199] [DEBUG] [app] Starting request: urn:request:f898cddb-892d-46d6-ad16-f44444e2c3cc (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:42:22,179 [197] [DEBUG] [app] Starting request: urn:request:9f447fac-46aa-488f-90d4-c5fe56841188 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:42:22,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:22,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:42:22,184 [209] [DEBUG] [app] Starting request: urn:request:cba4d566-2c7f-44ff-bee4-e6acbc8fbd93 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:42:22,185 [206] [DEBUG] [app] Starting request: urn:request:b09f1f73-1a23-46ee-98a7-34a2481882da (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:42:22,185 [209] [DEBUG] [app] Ending request: urn:request:cba4d566-2c7f-44ff-bee4-e6acbc8fbd93 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:cba4d566-2c7f-44ff-bee4-e6acbc8fbd93', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:42:22,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:42:22,185 [206] [DEBUG] [app] Ending request: urn:request:b09f1f73-1a23-46ee-98a7-34a2481882da (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b09f1f73-1a23-46ee-98a7-34a2481882da', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:22,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:42:22,186 [206] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:42:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:22,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:22,189 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:22,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:22,190 [200] [DEBUG] [app] Starting request: urn:request:7ff62f02-2430-4ad9-99f7-419648a433f8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:22,190 [199] [DEBUG] [app] Starting request: urn:request:b0ba15a6-2195-48bc-ac48-b7bfcb037489 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:22,191 [200] [DEBUG] [app] Ending request: urn:request:7ff62f02-2430-4ad9-99f7-419648a433f8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7ff62f02-2430-4ad9-99f7-419648a433f8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:22,191 [199] [DEBUG] [app] Ending request: urn:request:b0ba15a6-2195-48bc-ac48-b7bfcb037489 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b0ba15a6-2195-48bc-ac48-b7bfcb037489', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:22,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:42:22,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:42:22,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:22,191 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:22,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:42:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:42:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:42:22,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:42:22,202 [199] [DEBUG] [app] Ending request: urn:request:f898cddb-892d-46d6-ad16-f44444e2c3cc (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f898cddb-892d-46d6-ad16-f44444e2c3cc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:42:22,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:42:22,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:42:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:42:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:42:22,202 [197] [DEBUG] [app] Ending request: urn:request:9f447fac-46aa-488f-90d4-c5fe56841188 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9f447fac-46aa-488f-90d4-c5fe56841188', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:42:22,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:42:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:42:22,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:42:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:42:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:42:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:28.874906+00:00 (in 6.580635 seconds) chunkcleanupworker stdout | 2023-08-08 06:42:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:32 UTC)" (scheduled at 2023-08-08 06:42:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:42:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:42:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 22, 294607), True, datetime.datetime(2023, 8, 8, 6, 42, 22, 294607), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:42:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:42:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:42:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:32 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:42:22,640 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} gcworker stdout | 2023-08-08 06:42:25,337 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:42:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:42:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:35.371682+00:00 (in 9.999515 seconds) notificationworker stdout | 2023-08-08 06:42:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:35 UTC)" (scheduled at 2023-08-08 06:42:25.371682+00:00) notificationworker stdout | 2023-08-08 06:42:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:42:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 25, 372498), True, datetime.datetime(2023, 8, 8, 6, 42, 25, 372498), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:42:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:42:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:42:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:42:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:42:25,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:47.398551+00:00 (in 21.883718 seconds) exportactionlogsworker stdout | 2023-08-08 06:42:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:25 UTC)" (scheduled at 2023-08-08 06:42:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:42:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:42:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:25 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:42:25,618 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:42:25,621 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:42:25,625 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:42:26,707 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:42:26,716 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:42:26,716 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:42:26,720 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:42:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:42:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:49.687363+00:00 (in 22.796076 seconds) securityworker stdout | 2023-08-08 06:42:26,892 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:56 UTC)" (scheduled at 2023-08-08 06:42:26.890741+00:00) securityworker stdout | 2023-08-08 06:42:26,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:42:26,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:42:26,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:42:26,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:42:26,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:42:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:42:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:42:26,896 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:42:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:42:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:42:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 37, 26, 897592), 1, 2]) securityworker stdout | 2023-08-08 06:42:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:26,908 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 37, 26, 897592), 1, 2]) securityworker stdout | 2023-08-08 06:42:26,909 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:26,909 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:26,910 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:42:26,910 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:42:26,910 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:42:26,910 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:42:26,910 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:42:26,910 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:42:26,910 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:42:26,910 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:42:56 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:42:27,153 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:42:27,497 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:42:27,573 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:42:28,008 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:42:28,333 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:42:28,466 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:42:28,475 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:42:28,484 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:42:28,563 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} chunkcleanupworker stdout | 2023-08-08 06:42:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:42:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:32.293757+00:00 (in 3.418305 seconds) chunkcleanupworker stdout | 2023-08-08 06:42:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:28 UTC)" (scheduled at 2023-08-08 06:42:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:42:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:42:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:28 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 06:42:29,049 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:42:29,272 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:42:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:42:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:39.417660+00:00 (in 9.999493 seconds) gcworker stdout | 2023-08-08 06:42:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:39 UTC)" (scheduled at 2023-08-08 06:42:29.417660+00:00) gcworker stdout | 2023-08-08 06:42:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267349418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:42:29,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:42:29,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:42:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:42:30,671 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:42:30,817 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:42:31,845 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:42:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:42:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:42.293757+00:00 (in 9.999429 seconds) chunkcleanupworker stdout | 2023-08-08 06:42:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:42 UTC)" (scheduled at 2023-08-08 06:42:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:42:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:42:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 32, 294900), True, datetime.datetime(2023, 8, 8, 6, 42, 32, 294900), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:42:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:42:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:42:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:42:32,681 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:42:32,850 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:42:33,174 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:42:33,805 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:42:34,003 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:42:34,006 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:42:34,010 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:42:34,013 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:42:34,016 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:42:34,020 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:42:34,023 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:42:34,026 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:42:34,031 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} namespacegcworker stdout | 2023-08-08 06:42:34,071 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:42:34,569 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:42:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:42:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:45.371682+00:00 (in 9.999488 seconds) notificationworker stdout | 2023-08-08 06:42:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:45 UTC)" (scheduled at 2023-08-08 06:42:35.371682+00:00) notificationworker stdout | 2023-08-08 06:42:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:42:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 35, 372559), True, datetime.datetime(2023, 8, 8, 6, 42, 35, 372559), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:42:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:42:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:42:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:42:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:42:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:54.393627+00:00 (in 17.813277 seconds) namespacegcworker stdout | 2023-08-08 06:42:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:36 UTC)" (scheduled at 2023-08-08 06:42:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:42:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:42:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:36 UTC)" executed successfully builder stdout | 2023-08-08 06:42:36,721 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:42:36,730 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:42:36,730 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:42:36,734 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:42:37,179 [200] [DEBUG] [app] Starting request: urn:request:8b61d150-f22a-4ada-bc97-28389ba5c64d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:42:37,179 [198] [DEBUG] [app] Starting request: urn:request:f3bbe881-e8ea-4119-948b-1ca474ac36ae (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:42:37,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:37,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:37,184 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:42:37,185 [210] [DEBUG] [app] Starting request: urn:request:d25be278-d7e7-4537-8dcc-ba2fd436ace4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:42:37,185 [210] [DEBUG] [app] Ending request: urn:request:d25be278-d7e7-4537-8dcc-ba2fd436ace4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d25be278-d7e7-4537-8dcc-ba2fd436ace4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:42:37,185 [204] [DEBUG] [app] Starting request: urn:request:49cbe223-3f59-4b7a-8ae0-d330bfe23877 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:42:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:42:37,186 [204] [DEBUG] [app] Ending request: urn:request:49cbe223-3f59-4b7a-8ae0-d330bfe23877 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:49cbe223-3f59-4b7a-8ae0-d330bfe23877', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:37,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:42:37,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:42:37,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:37,188 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:37,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:37,190 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:37,191 [197] [DEBUG] [app] Starting request: urn:request:f1279478-f4ee-4329-9fc6-cae8d3b30648 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:37,191 [197] [DEBUG] [app] Ending request: urn:request:f1279478-f4ee-4329-9fc6-cae8d3b30648 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f1279478-f4ee-4329-9fc6-cae8d3b30648', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:42:37,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:42:37,191 [199] [DEBUG] [app] Starting request: urn:request:d169051a-d422-435c-ab2f-ef0e493d2696 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:42:37,192 [199] [DEBUG] [app] Ending request: urn:request:d169051a-d422-435c-ab2f-ef0e493d2696 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d169051a-d422-435c-ab2f-ef0e493d2696', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:37,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:42:37,193 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2023-08-08 06:42:37,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:42:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:42:37,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:42:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:42:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:07.197412+00:00 (in 29.999520 seconds) buildlogsarchiver stdout | 2023-08-08 06:42:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:07 UTC)" (scheduled at 2023-08-08 06:42:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:42:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 42, 37, 198292), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:42:37,202 [198] [DEBUG] [app] Ending request: urn:request:f3bbe881-e8ea-4119-948b-1ca474ac36ae (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f3bbe881-e8ea-4119-948b-1ca474ac36ae', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:42:37,203 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:42:37,203 [200] [DEBUG] [app] Ending request: urn:request:8b61d150-f22a-4ada-bc97-28389ba5c64d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8b61d150-f22a-4ada-bc97-28389ba5c64d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:42:37,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:42:37,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:42:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:42:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:42:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:42:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:42:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:42:37,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:42:37,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:42:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:42:37,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:42:37,209 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:42:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:42:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:49.417660+00:00 (in 9.999498 seconds) gcworker stdout | 2023-08-08 06:42:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:49 UTC)" (scheduled at 2023-08-08 06:42:39.417660+00:00) gcworker stdout | 2023-08-08 06:42:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267359418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:42:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:42:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:42:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:42:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:42:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:52.293757+00:00 (in 9.999479 seconds) chunkcleanupworker stdout | 2023-08-08 06:42:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:52 UTC)" (scheduled at 2023-08-08 06:42:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:42:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:42:42,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 42, 294970), True, datetime.datetime(2023, 8, 8, 6, 42, 42, 294970), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:42:42,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:42:42,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:42:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:42:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:42:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:55.371682+00:00 (in 9.999518 seconds) notificationworker stdout | 2023-08-08 06:42:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:55 UTC)" (scheduled at 2023-08-08 06:42:45.371682+00:00) notificationworker stdout | 2023-08-08 06:42:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:42:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 45, 372486), True, datetime.datetime(2023, 8, 8, 6, 42, 45, 372486), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:42:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:42:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:42:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:55 UTC)" executed successfully builder stdout | 2023-08-08 06:42:46,744 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:42:46,754 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:42:46,754 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:42:46,758 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:42:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:42:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:25.514341+00:00 (in 38.115291 seconds) exportactionlogsworker stdout | 2023-08-08 06:42:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:47 UTC)" (scheduled at 2023-08-08 06:42:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:42:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:42:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 47, 399527), True, datetime.datetime(2023, 8, 8, 6, 42, 47, 399527), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:42:47,407 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:42:47,407 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:42:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:42:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:42:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:59.417660+00:00 (in 9.999472 seconds) gcworker stdout | 2023-08-08 06:42:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:59 UTC)" (scheduled at 2023-08-08 06:42:49.417660+00:00) gcworker stdout | 2023-08-08 06:42:49,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267369419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:42:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:42:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:42:49,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:42:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:42:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:42:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:42:56.890741+00:00 (in 7.202851 seconds) securityworker stdout | 2023-08-08 06:42:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:19 UTC)" (scheduled at 2023-08-08 06:42:49.687363+00:00) securityworker stdout | 2023-08-08 06:42:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:42:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:42:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:42:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:42:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:42:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:42:50,007 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:42:51,004 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} manifestbackfillworker stdout | 2023-08-08 06:42:52,156 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:42:52,179 [197] [DEBUG] [app] Starting request: urn:request:e0ed6e26-568f-426c-b41c-b18ddfc3f6de (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:42:52,180 [200] [DEBUG] [app] Starting request: urn:request:8d0c1cdd-fd92-4a46-a91e-9f8442b12842 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:42:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:52,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:52,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:52,184 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:42:52,184 [203] [DEBUG] [app] Starting request: urn:request:86416f2e-2139-4e70-8487-b454d462072d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:42:52,185 [203] [DEBUG] [app] Ending request: urn:request:86416f2e-2139-4e70-8487-b454d462072d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:86416f2e-2139-4e70-8487-b454d462072d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:42:52,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:42:52,185 [210] [DEBUG] [app] Starting request: urn:request:10ecd867-8912-4758-aeac-495d91f4d488 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:52,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:42:52,186 [210] [DEBUG] [app] Ending request: urn:request:10ecd867-8912-4758-aeac-495d91f4d488 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:10ecd867-8912-4758-aeac-495d91f4d488', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:42:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:42:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:52,188 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:42:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:42:52,191 [199] [DEBUG] [app] Starting request: urn:request:b2a5b10d-94bd-4f3b-b5ef-aaf192cc2be9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:52,191 [198] [DEBUG] [app] Starting request: urn:request:60234705-bb6d-4de0-99f7-7324b6111ec6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:42:52,192 [199] [DEBUG] [app] Ending request: urn:request:b2a5b10d-94bd-4f3b-b5ef-aaf192cc2be9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b2a5b10d-94bd-4f3b-b5ef-aaf192cc2be9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:42:52,192 [198] [DEBUG] [app] Ending request: urn:request:60234705-bb6d-4de0-99f7-7324b6111ec6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:60234705-bb6d-4de0-99f7-7324b6111ec6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:42:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:42:52,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:42:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:42:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:42:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:42:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:42:52,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:42:52,198 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:42:52,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:42:52,203 [200] [DEBUG] [app] Ending request: urn:request:8d0c1cdd-fd92-4a46-a91e-9f8442b12842 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8d0c1cdd-fd92-4a46-a91e-9f8442b12842', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:42:52,203 [200] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:42:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:42:52,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:42:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:42:52,203 [197] [DEBUG] [app] Ending request: urn:request:e0ed6e26-568f-426c-b41c-b18ddfc3f6de (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e0ed6e26-568f-426c-b41c-b18ddfc3f6de', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:42:52,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:42:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) gunicorn-web stdout | 2023-08-08 06:42:52,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:42:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:42:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:42:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:02.293757+00:00 (in 9.999449 seconds) chunkcleanupworker stdout | 2023-08-08 06:42:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:02 UTC)" (scheduled at 2023-08-08 06:42:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:42:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:42:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 52, 294866), True, datetime.datetime(2023, 8, 8, 6, 42, 52, 294866), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:42:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:42:52,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:42:52,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:02 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:42:52,673 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:42:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:42:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:13.097375+00:00 (in 18.812685 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:42:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:54 UTC)" (scheduled at 2023-08-08 06:42:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:42:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:42:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:42:54,393 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:42:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:36.579841+00:00 (in 42.185665 seconds) namespacegcworker stdout | 2023-08-08 06:42:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:54 UTC)" (scheduled at 2023-08-08 06:42:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:42:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:42:54,396 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 54, 394829), True, datetime.datetime(2023, 8, 8, 6, 42, 54, 394829), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:42:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:42:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:42:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:43:54 UTC)" executed successfully gcworker stdout | 2023-08-08 06:42:55,369 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} notificationworker stdout | 2023-08-08 06:42:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:42:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:02.178530+00:00 (in 6.806400 seconds) notificationworker stdout | 2023-08-08 06:42:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:05 UTC)" (scheduled at 2023-08-08 06:42:55.371682+00:00) notificationworker stdout | 2023-08-08 06:42:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:42:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 42, 55, 372694), True, datetime.datetime(2023, 8, 8, 6, 42, 55, 372694), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:42:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:42:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:42:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:05 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:42:55,628 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:42:55,631 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:42:55,633 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:42:56,768 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:42:56,777 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:42:56,777 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:42:56,781 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:42:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:42:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:19.687363+00:00 (in 22.796028 seconds) securityworker stdout | 2023-08-08 06:42:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:26 UTC)" (scheduled at 2023-08-08 06:42:56.890741+00:00) securityworker stdout | 2023-08-08 06:42:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:42:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:42:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:42:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:42:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:42:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:42:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:42:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:42:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:42:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:42:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 37, 56, 896867), 1, 2]) securityworker stdout | 2023-08-08 06:42:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 37, 56, 896867), 1, 2]) securityworker stdout | 2023-08-08 06:42:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:42:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:42:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:42:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:42:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:42:56,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:42:56,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:42:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:42:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:42:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:26 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:42:57,190 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:42:57,514 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:42:57,610 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:42:58,018 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:42:58,348 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:42:58,480 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:42:58,489 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:42:58,499 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:42:58,587 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} repositoryactioncounter stdout | 2023-08-08 06:42:59,077 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:42:59,295 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:42:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:42:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:09.417660+00:00 (in 9.999003 seconds) gcworker stdout | 2023-08-08 06:42:59,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:09 UTC)" (scheduled at 2023-08-08 06:42:59.417660+00:00) gcworker stdout | 2023-08-08 06:42:59,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267379419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:42:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:42:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:42:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:43:00,689 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:43:00,855 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:43:01,857 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:43:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:43:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:05.371682+00:00 (in 3.192620 seconds) notificationworker stdout | 2023-08-08 06:43:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:02 UTC)" (scheduled at 2023-08-08 06:43:02.178530+00:00) notificationworker stdout | 2023-08-08 06:43:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:43:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:43:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:43:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:12.293757+00:00 (in 9.999518 seconds) chunkcleanupworker stdout | 2023-08-08 06:43:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:12 UTC)" (scheduled at 2023-08-08 06:43:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:43:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:43:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 2, 294859), True, datetime.datetime(2023, 8, 8, 6, 43, 2, 294859), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:43:02,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:43:02,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:43:02,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:43:02,714 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:43:02,887 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:43:03,197 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:43:03,813 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:43:04,015 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:43:04,021 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:43:04,026 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:43:04,027 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:43:04,030 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:43:04,033 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:43:04,036 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:43:04,041 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:43:04,045 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} namespacegcworker stdout | 2023-08-08 06:43:04,109 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} repositorygcworker stdout | 2023-08-08 06:43:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:43:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:09.793658+00:00 (in 5.605744 seconds) repositorygcworker stdout | 2023-08-08 06:43:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:04 UTC)" (scheduled at 2023-08-08 06:43:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:43:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:43:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:04 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:43:04,606 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:43:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:43:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:15.371682+00:00 (in 9.999497 seconds) notificationworker stdout | 2023-08-08 06:43:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:15 UTC)" (scheduled at 2023-08-08 06:43:05.371682+00:00) notificationworker stdout | 2023-08-08 06:43:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:43:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 5, 372607), True, datetime.datetime(2023, 8, 8, 6, 43, 5, 372607), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:43:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:43:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:43:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:15 UTC)" executed successfully builder stdout | 2023-08-08 06:43:06,782 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:43:06,790 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:43:06,790 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:43:06,794 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:43:07,179 [199] [DEBUG] [app] Starting request: urn:request:3dc28663-75d9-413b-adaf-d914370916f9 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:43:07,179 [197] [DEBUG] [app] Starting request: urn:request:37062181-17e8-4256-a860-123258243955 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:43:07,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:07,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:43:07,184 [205] [DEBUG] [app] Starting request: urn:request:e92c5bd8-f545-4e00-8fe8-86657478b794 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:43:07,185 [209] [DEBUG] [app] Starting request: urn:request:de90bbd3-15a9-429d-a06e-fbc2073c52f2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:43:07,185 [205] [DEBUG] [app] Ending request: urn:request:e92c5bd8-f545-4e00-8fe8-86657478b794 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e92c5bd8-f545-4e00-8fe8-86657478b794', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:43:07,185 [209] [DEBUG] [app] Ending request: urn:request:de90bbd3-15a9-429d-a06e-fbc2073c52f2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:de90bbd3-15a9-429d-a06e-fbc2073c52f2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:43:07,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:43:07,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-registry stdout | 2023-08-08 06:43:07,186 [205] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:43:07,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:07,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:07,193 [198] [DEBUG] [app] Starting request: urn:request:033dd089-35ff-4b41-b7ca-f03f3a5d1e16 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:43:07,193 [197] [DEBUG] [app] Starting request: urn:request:26e147ab-6f7c-41a4-a7ce-741126b721b9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:43:07,193 [198] [DEBUG] [app] Ending request: urn:request:033dd089-35ff-4b41-b7ca-f03f3a5d1e16 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:033dd089-35ff-4b41-b7ca-f03f3a5d1e16', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:43:07,193 [197] [DEBUG] [app] Ending request: urn:request:26e147ab-6f7c-41a4-a7ce-741126b721b9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:26e147ab-6f7c-41a4-a7ce-741126b721b9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:43:07,194 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:43:07,194 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:43:07,194 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:07,194 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:07,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:43:07,194 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:43:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:43:07,197 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:37.197412+00:00 (in 29.999570 seconds) buildlogsarchiver stdout | 2023-08-08 06:43:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:37 UTC)" (scheduled at 2023-08-08 06:43:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:43:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 43, 7, 198276), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:43:07,199 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:43:07,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:43:07,204 [197] [DEBUG] [app] Ending request: urn:request:37062181-17e8-4256-a860-123258243955 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:37062181-17e8-4256-a860-123258243955', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:43:07,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:43:07,204 [199] [DEBUG] [app] Ending request: urn:request:3dc28663-75d9-413b-adaf-d914370916f9 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3dc28663-75d9-413b-adaf-d914370916f9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:43:07,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:43:07,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:43:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:43:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) gunicorn-web stdout | 2023-08-08 06:43:07,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:43:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:43:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.027 47 0.027) buildlogsarchiver stdout | 2023-08-08 06:43:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:43:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:43:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:43:07,211 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:43:07,211 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:43:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:43:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:19.417660+00:00 (in 9.999397 seconds) gcworker stdout | 2023-08-08 06:43:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:19 UTC)" (scheduled at 2023-08-08 06:43:09.417660+00:00) gcworker stdout | 2023-08-08 06:43:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267389418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:43:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:43:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:43:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:43:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:43:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:04.187429+00:00 (in 54.393220 seconds) repositorygcworker stdout | 2023-08-08 06:43:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:09 UTC)" (scheduled at 2023-08-08 06:43:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:43:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:43:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 9, 794584), True, datetime.datetime(2023, 8, 8, 6, 43, 9, 794584), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:43:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:43:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:43:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:43:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:43:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:22.293757+00:00 (in 9.999446 seconds) chunkcleanupworker stdout | 2023-08-08 06:43:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:22 UTC)" (scheduled at 2023-08-08 06:43:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:43:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:43:12,296 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 12, 294981), True, datetime.datetime(2023, 8, 8, 6, 43, 12, 294981), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:43:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:43:12,304 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:43:12,304 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:43:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:43:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:54.284093+00:00 (in 41.186202 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:43:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:13 UTC)" (scheduled at 2023-08-08 06:43:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:43:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:43:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 13, 98380), True, datetime.datetime(2023, 8, 8, 6, 43, 13, 98380), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:43:13,106 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:43:13,107 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:43:13,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:43:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:43:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:25.371682+00:00 (in 9.999516 seconds) notificationworker stdout | 2023-08-08 06:43:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:25 UTC)" (scheduled at 2023-08-08 06:43:15.371682+00:00) notificationworker stdout | 2023-08-08 06:43:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:43:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 15, 372804), True, datetime.datetime(2023, 8, 8, 6, 43, 15, 372804), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:43:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:43:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:43:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:25 UTC)" executed successfully builder stdout | 2023-08-08 06:43:16,805 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:43:16,813 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:43:16,813 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:43:16,817 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:43:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:43:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:29.417660+00:00 (in 9.999321 seconds) gcworker stdout | 2023-08-08 06:43:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:29 UTC)" (scheduled at 2023-08-08 06:43:19.417660+00:00) gcworker stdout | 2023-08-08 06:43:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267399418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:43:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:43:19,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:43:19,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:43:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:43:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:26.890741+00:00 (in 7.202879 seconds) securityworker stdout | 2023-08-08 06:43:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:49 UTC)" (scheduled at 2023-08-08 06:43:19.687363+00:00) securityworker stdout | 2023-08-08 06:43:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:43:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:43:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:43:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:43:19,698 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:43:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:43:19,701 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:19,701 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 38, 19, 692095), 1, 2]) securityworker stdout | 2023-08-08 06:43:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 38, 19, 692095), 1, 2]) securityworker stdout | 2023-08-08 06:43:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:43:19,704 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:49 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:43:20,016 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:43:21,028 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:43:22,179 [198] [DEBUG] [app] Starting request: urn:request:0026def2-9d06-4397-b0bb-7649bcec485b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:43:22,180 [199] [DEBUG] [app] Starting request: urn:request:0417d49c-adac-47e1-96f5-1cb1eb502ccb (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:43:22,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:22,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:43:22,184 [208] [DEBUG] [app] Starting request: urn:request:d89a090c-7eb6-4373-8787-c654ad993b78 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:43:22,185 [208] [DEBUG] [app] Ending request: urn:request:d89a090c-7eb6-4373-8787-c654ad993b78 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d89a090c-7eb6-4373-8787-c654ad993b78', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:43:22,185 [204] [DEBUG] [app] Starting request: urn:request:1c548412-d8cf-41e9-be4e-7c18306427db (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:43:22,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:43:22,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:43:22,185 [204] [DEBUG] [app] Ending request: urn:request:1c548412-d8cf-41e9-be4e-7c18306427db (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1c548412-d8cf-41e9-be4e-7c18306427db', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:43:22,186 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:43:22,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:22,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:22,190 [199] [DEBUG] [app] Starting request: urn:request:6b7a8c10-a764-42ac-add9-f6ef0b5f01c9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:43:22,191 [199] [DEBUG] [app] Ending request: urn:request:6b7a8c10-a764-42ac-add9-f6ef0b5f01c9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6b7a8c10-a764-42ac-add9-f6ef0b5f01c9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:43:22,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:43:22,191 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:22,192 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:22,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:43:22,192 [199] [DEBUG] [app] Starting request: urn:request:47c37039-9328-41c0-bfd9-211737e4147c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:43:22,193 [199] [DEBUG] [app] Ending request: urn:request:47c37039-9328-41c0-bfd9-211737e4147c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:47c37039-9328-41c0-bfd9-211737e4147c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:43:22,193 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:43:22,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:22,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql manifestbackfillworker stdout | 2023-08-08 06:43:22,194 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} gunicorn-web stdout | 2023-08-08 06:43:22,196 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:43:22,198 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:43:22,202 [198] [DEBUG] [app] Ending request: urn:request:0026def2-9d06-4397-b0bb-7649bcec485b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0026def2-9d06-4397-b0bb-7649bcec485b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:43:22,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:43:22,202 [199] [DEBUG] [app] Ending request: urn:request:0417d49c-adac-47e1-96f5-1cb1eb502ccb (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0417d49c-adac-47e1-96f5-1cb1eb502ccb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:43:22,202 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:43:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:43:22,202 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:43:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:43:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.023 47 0.024) gunicorn-web stdout | 2023-08-08 06:43:22,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:43:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" chunkcleanupworker stdout | 2023-08-08 06:43:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:43:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:28.874906+00:00 (in 6.580587 seconds) chunkcleanupworker stdout | 2023-08-08 06:43:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:32 UTC)" (scheduled at 2023-08-08 06:43:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:43:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:43:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 22, 294673), True, datetime.datetime(2023, 8, 8, 6, 43, 22, 294673), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:43:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:43:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:43:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:32 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:43:22,712 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} notificationworker stdout | 2023-08-08 06:43:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:43:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:35.371682+00:00 (in 9.999480 seconds) notificationworker stdout | 2023-08-08 06:43:25,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:35 UTC)" (scheduled at 2023-08-08 06:43:25.371682+00:00) notificationworker stdout | 2023-08-08 06:43:25,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:43:25,374 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 25, 373244), True, datetime.datetime(2023, 8, 8, 6, 43, 25, 373244), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:43:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:43:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:43:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:35 UTC)" executed successfully gcworker stdout | 2023-08-08 06:43:25,406 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} exportactionlogsworker stdout | 2023-08-08 06:43:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:43:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:33.514094+00:00 (in 7.999216 seconds) exportactionlogsworker stdout | 2023-08-08 06:43:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:25 UTC)" (scheduled at 2023-08-08 06:43:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:43:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:43:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:25 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:43:25,637 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:43:25,641 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:43:25,643 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:43:26,827 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:43:26,836 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:43:26,836 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:43:26,840 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:43:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:43:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:49.687363+00:00 (in 22.796082 seconds) securityworker stdout | 2023-08-08 06:43:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:56 UTC)" (scheduled at 2023-08-08 06:43:26.890741+00:00) securityworker stdout | 2023-08-08 06:43:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:43:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:43:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:43:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:43:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:43:26,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:43:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:43:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:43:26,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:43:26,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:43:26,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:26,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 38, 26, 897281), 1, 2]) securityworker stdout | 2023-08-08 06:43:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:26,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 38, 26, 897281), 1, 2]) securityworker stdout | 2023-08-08 06:43:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:26,909 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:43:26,909 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:43:26,909 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:43:26,909 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:43:26,909 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:43:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:43:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:43:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:43:56 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:43:27,227 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:43:27,551 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:43:27,618 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:43:28,025 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:43:28,361 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:43:28,494 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:43:28,502 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:43:28,512 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:43:28,618 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} chunkcleanupworker stdout | 2023-08-08 06:43:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:43:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:32.293757+00:00 (in 3.418368 seconds) chunkcleanupworker stdout | 2023-08-08 06:43:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:28 UTC)" (scheduled at 2023-08-08 06:43:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:43:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:43:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:28 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 06:43:29,115 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:43:29,314 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:43:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:43:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:39.417660+00:00 (in 9.999507 seconds) gcworker stdout | 2023-08-08 06:43:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:39 UTC)" (scheduled at 2023-08-08 06:43:29.417660+00:00) gcworker stdout | 2023-08-08 06:43:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267409418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:43:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:43:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:43:29,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:43:30,727 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:43:30,893 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:43:31,894 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:43:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:43:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:42.293757+00:00 (in 9.999422 seconds) chunkcleanupworker stdout | 2023-08-08 06:43:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:42 UTC)" (scheduled at 2023-08-08 06:43:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:43:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:43:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 32, 294713), True, datetime.datetime(2023, 8, 8, 6, 43, 32, 294713), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:43:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:43:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:43:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:43:32,723 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:43:32,916 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:43:33,205 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} exportactionlogsworker stdout | 2023-08-08 06:43:33,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:43:33,514 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:47.398551+00:00 (in 13.883965 seconds) exportactionlogsworker stdout | 2023-08-08 06:43:33,514 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:48:33 UTC)" (scheduled at 2023-08-08 06:43:33.514094+00:00) exportactionlogsworker stdout | 2023-08-08 06:43:33,515 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 43, 33, 514909), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:43:33,522 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 43, 33, 514909), True, datetime.datetime(2023, 8, 8, 6, 43, 33, 514909), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:43:33,524 [60] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 43, 33, 514909), True, datetime.datetime(2023, 8, 8, 6, 43, 33, 514909), 0, 'exportactionlogs/%', False, datetime.datetime(2023, 8, 8, 6, 43, 33, 514909), 'exportactionlogs/%']) exportactionlogsworker stdout | 2023-08-08 06:43:33,525 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:43:33,525 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:48:33 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:43:33,848 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:43:34,025 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:43:34,029 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:43:34,034 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:43:34,038 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:43:34,043 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:43:34,045 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:43:34,049 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:43:34,052 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:43:34,055 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} namespacegcworker stdout | 2023-08-08 06:43:34,132 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:43:34,645 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:43:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:43:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:45.371682+00:00 (in 9.999491 seconds) notificationworker stdout | 2023-08-08 06:43:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:45 UTC)" (scheduled at 2023-08-08 06:43:35.371682+00:00) notificationworker stdout | 2023-08-08 06:43:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:43:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 35, 372703), True, datetime.datetime(2023, 8, 8, 6, 43, 35, 372703), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:43:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:43:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:43:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:43:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:43:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:54.393627+00:00 (in 17.813287 seconds) namespacegcworker stdout | 2023-08-08 06:43:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:36 UTC)" (scheduled at 2023-08-08 06:43:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:43:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:43:36,580 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:36 UTC)" executed successfully builder stdout | 2023-08-08 06:43:36,850 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:43:36,859 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:43:36,859 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:43:36,863 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:43:37,179 [197] [DEBUG] [app] Starting request: urn:request:b06453fc-d7ca-471d-81c4-b0f229ec513e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:43:37,179 [200] [DEBUG] [app] Starting request: urn:request:ef5c5a69-6b8c-48b8-a6ba-ef93a038a9ad (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:43:37,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:43:37,184 [203] [DEBUG] [app] Starting request: urn:request:7610f7d2-e2b1-4d8e-a422-ce61b6231582 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:43:37,185 [203] [DEBUG] [app] Ending request: urn:request:7610f7d2-e2b1-4d8e-a422-ce61b6231582 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7610f7d2-e2b1-4d8e-a422-ce61b6231582', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:43:37,185 [209] [DEBUG] [app] Starting request: urn:request:68a549a6-371e-4521-b671-5bc5662179c4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:43:37,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:43:37,185 [209] [DEBUG] [app] Ending request: urn:request:68a549a6-371e-4521-b671-5bc5662179c4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:68a549a6-371e-4521-b671-5bc5662179c4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:43:37,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:43:37,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:43:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:37,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:37,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:37,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:37,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:37,190 [199] [DEBUG] [app] Starting request: urn:request:3ddea29a-1960-47ce-8752-e21496a8bf12 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:43:37,191 [198] [DEBUG] [app] Starting request: urn:request:999ecefd-a723-4915-8290-1dd232e7d978 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:43:37,191 [199] [DEBUG] [app] Ending request: urn:request:3ddea29a-1960-47ce-8752-e21496a8bf12 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3ddea29a-1960-47ce-8752-e21496a8bf12', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:43:37,191 [198] [DEBUG] [app] Ending request: urn:request:999ecefd-a723-4915-8290-1dd232e7d978 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:999ecefd-a723-4915-8290-1dd232e7d978', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:43:37,191 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:43:37,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:43:37,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:37,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:43:37,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:43:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:43:37,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:43:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:43:37,197 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:07.197412+00:00 (in 29.999572 seconds) buildlogsarchiver stdout | 2023-08-08 06:43:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:07 UTC)" (scheduled at 2023-08-08 06:43:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:43:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 43, 37, 198330), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:43:37,202 [197] [DEBUG] [app] Ending request: urn:request:b06453fc-d7ca-471d-81c4-b0f229ec513e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b06453fc-d7ca-471d-81c4-b0f229ec513e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:43:37,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:43:37,202 [200] [DEBUG] [app] Ending request: urn:request:ef5c5a69-6b8c-48b8-a6ba-ef93a038a9ad (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ef5c5a69-6b8c-48b8-a6ba-ef93a038a9ad', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:43:37,203 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:43:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:43:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:43:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:43:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) gunicorn-web stdout | 2023-08-08 06:43:37,203 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:43:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:43:37,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:43:37,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:43:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:43:37,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:43:37,209 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:43:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:43:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:49.417660+00:00 (in 9.999479 seconds) gcworker stdout | 2023-08-08 06:43:39,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:49 UTC)" (scheduled at 2023-08-08 06:43:39.417660+00:00) gcworker stdout | 2023-08-08 06:43:39,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267419419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:43:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:43:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:43:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:49 UTC)" executed successfully gunicorn-web stdout | 2023-08-08 06:43:40,323 [198] [DEBUG] [app] Starting request: urn:request:f16c5820-6681-4bf2-9936-bfb7cacafa52 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:43:40,323 [199] [DEBUG] [app] Starting request: urn:request:b0335d0b-d968-4f00-81c1-21fa100fcef7 (/api/v1/user/notifications) {'X-Forwarded-For': '10.130.2.1'} gunicorn-web stdout | 2023-08-08 06:43:40,323 [198] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:43:40,323 [199] [DEBUG] [app] User loader loading deferred user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:43:40,324 [198] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:43:40,324 [199] [DEBUG] [auth.cookie] Loading user from cookie: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:43:40,324 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:43:40,324 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['506693df-aa05-419c-aa63-3bfe60e4edd1', False, 1, 0]) gunicorn-web stdout | 2023-08-08 06:43:40,331 [199] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:43:40,331 [198] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2023-08-08 06:43:40,331 [199] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:43:40,331 [198] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2023-08-08 06:43:40,331 [199] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:43:40,332 [198] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:43:40,332 [199] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:43:40,332 [198] [DEBUG] [endpoints.api] Checking permission for user user1 gunicorn-web stdout | 2023-08-08 06:43:40,332 [199] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:43:40,332 [198] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 506693df-aa05-419c-aa63-3bfe60e4edd1 gunicorn-web stdout | 2023-08-08 06:43:40,332 [199] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:43:40,332 [198] [DEBUG] [auth.permissions] User permission: _UserTypeNeed(type='user', username='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:43:40,332 [199] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:43:40,332 [198] [DEBUG] [auth.permissions] User namespace permission: _NamespaceWideNeed(type='organization', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:43:40,332 [199] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:43:40,332 [198] [DEBUG] [auth.permissions] User namespace repo permission: _NamespaceWideNeed(type='organizationrepo', namespace='user1', role='admin') gunicorn-web stdout | 2023-08-08 06:43:40,332 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [3, datetime.datetime(2023, 8, 11, 1, 55, 40, 332502), datetime.datetime(2023, 8, 8, 6, 43, 40, 332556)]) gunicorn-web stdout | 2023-08-08 06:43:40,333 [199] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 3, False, 3, 'admin', 6]) gunicorn-web stdout | 2023-08-08 06:43:40,334 [198] [DEBUG] [app] Ending request: urn:request:f16c5820-6681-4bf2-9936-bfb7cacafa52 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:f16c5820-6681-4bf2-9936-bfb7cacafa52', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:43:40,335 [198] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:43:40 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/2.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.013 62 0.014) gunicorn-web stdout | 2023-08-08 06:43:40,336 [198] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:43:40 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" gunicorn-web stdout | 2023-08-08 06:43:40,338 [199] [DEBUG] [app] Ending request: urn:request:b0335d0b-d968-4f00-81c1-21fa100fcef7 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:b0335d0b-d968-4f00-81c1-21fa100fcef7', 'remote_addr': '10.130.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0'} gunicorn-web stdout | 2023-08-08 06:43:40,339 [199] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 (-) - - [08/Aug/2023:06:43:40 +0000] "GET /api/v1/user/notifications HTTP/2.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" (0.017 878 0.017) gunicorn-web stdout | 2023-08-08 06:43:40,339 [199] [INFO] [gunicorn.access] 10.130.2.1 - - [08/Aug/2023:06:43:40 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu413osp03.qe.devcluster.openshift.com/repository/user1_build_test/gitlab/trigger/879924fa-0bc9-4217-93fe-9bfc73feaf0e" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" chunkcleanupworker stdout | 2023-08-08 06:43:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:43:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:52.293757+00:00 (in 9.999432 seconds) chunkcleanupworker stdout | 2023-08-08 06:43:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:52 UTC)" (scheduled at 2023-08-08 06:43:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:43:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:43:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 42, 294716), True, datetime.datetime(2023, 8, 8, 6, 43, 42, 294716), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:43:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:43:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:43:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:43:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:43:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:49.178248+00:00 (in 3.806064 seconds) notificationworker stdout | 2023-08-08 06:43:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:55 UTC)" (scheduled at 2023-08-08 06:43:45.371682+00:00) notificationworker stdout | 2023-08-08 06:43:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:43:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 45, 372750), True, datetime.datetime(2023, 8, 8, 6, 43, 45, 372750), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:43:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:43:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:43:45,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:55 UTC)" executed successfully builder stdout | 2023-08-08 06:43:46,872 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:43:46,881 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:43:46,881 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:43:46,885 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:43:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:43:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:25.514341+00:00 (in 38.115274 seconds) exportactionlogsworker stdout | 2023-08-08 06:43:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:47 UTC)" (scheduled at 2023-08-08 06:43:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:43:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:43:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 47, 399448), True, datetime.datetime(2023, 8, 8, 6, 43, 47, 399448), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:43:47,406 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:43:47,406 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:43:47,407 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:47 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:43:49,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:43:49,178 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:55.371682+00:00 (in 6.192865 seconds) notificationworker stdout | 2023-08-08 06:43:49,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:48:49 UTC)" (scheduled at 2023-08-08 06:43:49.178248+00:00) notificationworker stdout | 2023-08-08 06:43:49,179 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 43, 49, 179231), 'notification/%']) notificationworker stdout | 2023-08-08 06:43:49,186 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 43, 49, 179231), True, datetime.datetime(2023, 8, 8, 6, 43, 49, 179231), 0, 'notification/%']) notificationworker stdout | 2023-08-08 06:43:49,188 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 43, 49, 179231), True, datetime.datetime(2023, 8, 8, 6, 43, 49, 179231), 0, 'notification/%', False, datetime.datetime(2023, 8, 8, 6, 43, 49, 179231), 'notification/%']) notificationworker stdout | 2023-08-08 06:43:49,189 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:43:49,189 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:48:49 UTC)" executed successfully gcworker stdout | 2023-08-08 06:43:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:43:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:59.417660+00:00 (in 9.999470 seconds) gcworker stdout | 2023-08-08 06:43:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:59 UTC)" (scheduled at 2023-08-08 06:43:49.417660+00:00) gcworker stdout | 2023-08-08 06:43:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267429418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:43:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:43:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:43:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:43:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:43:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:43:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:43:56.890741+00:00 (in 7.202806 seconds) securityworker stdout | 2023-08-08 06:43:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:19 UTC)" (scheduled at 2023-08-08 06:43:49.687363+00:00) securityworker stdout | 2023-08-08 06:43:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:43:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:43:49,691 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:43:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:43:49,698 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:43:49,698 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:43:50,054 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:43:51,069 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:43:52,179 [197] [DEBUG] [app] Starting request: urn:request:106414db-bdad-472a-9d9c-c5793b7adc62 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:43:52,180 [199] [DEBUG] [app] Starting request: urn:request:bf81b79b-fc07-4847-a8ba-a38ec3fdc395 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:43:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:52,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:52,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:43:52,185 [209] [DEBUG] [app] Starting request: urn:request:70ce201e-395d-43d3-a726-c3d7f8f25aee (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:43:52,185 [210] [DEBUG] [app] Starting request: urn:request:61f4d07c-5e87-46ab-acef-000dca0d3383 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:43:52,185 [209] [DEBUG] [app] Ending request: urn:request:70ce201e-395d-43d3-a726-c3d7f8f25aee (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:70ce201e-395d-43d3-a726-c3d7f8f25aee', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:43:52,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:43:52,186 [210] [DEBUG] [app] Ending request: urn:request:61f4d07c-5e87-46ab-acef-000dca0d3383 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:61f4d07c-5e87-46ab-acef-000dca0d3383', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:43:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:43:52,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-web stdout | 2023-08-08 06:43:52,186 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:52,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:43:52,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:52,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:43:52,191 [198] [DEBUG] [app] Starting request: urn:request:af9d9e6f-213f-4d72-8a74-8453c5235c4a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:43:52,191 [197] [DEBUG] [app] Starting request: urn:request:8a310ae9-d0fe-48ec-a02e-e35c9d098591 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:43:52,191 [198] [DEBUG] [app] Ending request: urn:request:af9d9e6f-213f-4d72-8a74-8453c5235c4a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:af9d9e6f-213f-4d72-8a74-8453c5235c4a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:43:52,191 [197] [DEBUG] [app] Ending request: urn:request:8a310ae9-d0fe-48ec-a02e-e35c9d098591 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8a310ae9-d0fe-48ec-a02e-e35c9d098591', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:43:52,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:43:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:43:52,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:43:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:43:52,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:43:52,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:43:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:43:52,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:43:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:43:52,202 [199] [DEBUG] [app] Ending request: urn:request:bf81b79b-fc07-4847-a8ba-a38ec3fdc395 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bf81b79b-fc07-4847-a8ba-a38ec3fdc395', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:43:52,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:43:52,203 [197] [DEBUG] [app] Ending request: urn:request:106414db-bdad-472a-9d9c-c5793b7adc62 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:106414db-bdad-472a-9d9c-c5793b7adc62', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:43:52,203 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:43:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:43:52,203 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:43:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:43:52,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:43:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:43:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) manifestbackfillworker stdout | 2023-08-08 06:43:52,231 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} chunkcleanupworker stdout | 2023-08-08 06:43:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:43:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:02.293757+00:00 (in 9.999444 seconds) chunkcleanupworker stdout | 2023-08-08 06:43:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:02 UTC)" (scheduled at 2023-08-08 06:43:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:43:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:43:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 52, 294700), True, datetime.datetime(2023, 8, 8, 6, 43, 52, 294700), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:43:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:43:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:43:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:02 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:43:52,751 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:43:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:43:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:13.097375+00:00 (in 18.812632 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:43:54,285 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:54 UTC)" (scheduled at 2023-08-08 06:43:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:43:54,285 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:43:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:43:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:43:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:36.579841+00:00 (in 42.185670 seconds) namespacegcworker stdout | 2023-08-08 06:43:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:54 UTC)" (scheduled at 2023-08-08 06:43:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:43:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:43:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 54, 394649), True, datetime.datetime(2023, 8, 8, 6, 43, 54, 394649), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:43:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:43:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:43:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:44:54 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:43:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:43:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:02.178530+00:00 (in 6.806354 seconds) notificationworker stdout | 2023-08-08 06:43:55,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:05 UTC)" (scheduled at 2023-08-08 06:43:55.371682+00:00) notificationworker stdout | 2023-08-08 06:43:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:43:55,375 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 43, 55, 373828), True, datetime.datetime(2023, 8, 8, 6, 43, 55, 373828), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:43:55,382 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:43:55,382 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:43:55,382 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:05 UTC)" executed successfully gcworker stdout | 2023-08-08 06:43:55,445 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:43:55,646 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:43:55,651 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:43:55,654 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} builder stdout | 2023-08-08 06:43:56,889 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue securityworker stdout | 2023-08-08 06:43:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:43:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:19.687363+00:00 (in 22.795626 seconds) securityworker stdout | 2023-08-08 06:43:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:26 UTC)" (scheduled at 2023-08-08 06:43:56.890741+00:00) securityworker stdout | 2023-08-08 06:43:56,892 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:43:56,892 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:43:56,893 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:43:56,893 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:43:56,893 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:43:56,893 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:43:56,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:43:56,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:43:56,897 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) builder stdout | 2023-08-08 06:43:56,898 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:43:56,898 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:43:56,902 [55] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:43:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:43:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:56,905 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 38, 56, 897298), 1, 2]) securityworker stdout | 2023-08-08 06:43:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 38, 56, 897298), 1, 2]) securityworker stdout | 2023-08-08 06:43:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:43:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:43:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:43:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:43:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:43:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:43:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:43:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:43:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:43:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:26 UTC)" executed successfully blobuploadcleanupworker stdout | 2023-08-08 06:43:57,264 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:43:57,588 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:43:57,655 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:43:58,033 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:43:58,375 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:43:58,509 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:43:58,515 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:43:58,526 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:43:58,634 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} repositoryactioncounter stdout | 2023-08-08 06:43:59,136 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:43:59,352 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:43:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:43:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:09.417660+00:00 (in 9.999500 seconds) gcworker stdout | 2023-08-08 06:43:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:09 UTC)" (scheduled at 2023-08-08 06:43:59.417660+00:00) gcworker stdout | 2023-08-08 06:43:59,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267439418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:43:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:43:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:43:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:44:00,743 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:44:00,931 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:44:01,931 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:44:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:44:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:05.371682+00:00 (in 3.192669 seconds) notificationworker stdout | 2023-08-08 06:44:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:02 UTC)" (scheduled at 2023-08-08 06:44:02.178530+00:00) notificationworker stdout | 2023-08-08 06:44:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:44:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:44:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:44:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:12.293757+00:00 (in 9.999403 seconds) chunkcleanupworker stdout | 2023-08-08 06:44:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:12 UTC)" (scheduled at 2023-08-08 06:44:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:44:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:44:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 2, 294781), True, datetime.datetime(2023, 8, 8, 6, 44, 2, 294781), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:44:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:44:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:44:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:44:02,761 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:44:02,936 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:44:03,236 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:44:03,878 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:44:04,034 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:44:04,037 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:44:04,043 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:44:04,047 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:44:04,050 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:44:04,053 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:44:04,058 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:44:04,061 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:44:04,066 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} namespacegcworker stdout | 2023-08-08 06:44:04,170 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} repositorygcworker stdout | 2023-08-08 06:44:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:44:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:09.793658+00:00 (in 5.605731 seconds) repositorygcworker stdout | 2023-08-08 06:44:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:04 UTC)" (scheduled at 2023-08-08 06:44:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:44:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:44:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:04 UTC)" executed successfully expiredappspecifictokenworker stdout | 2023-08-08 06:44:04,683 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:44:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:44:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:15.371682+00:00 (in 9.999470 seconds) notificationworker stdout | 2023-08-08 06:44:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:15 UTC)" (scheduled at 2023-08-08 06:44:05.371682+00:00) notificationworker stdout | 2023-08-08 06:44:05,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:44:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 5, 373093), True, datetime.datetime(2023, 8, 8, 6, 44, 5, 373093), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:44:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:44:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:44:05,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:15 UTC)" executed successfully builder stdout | 2023-08-08 06:44:06,909 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:44:06,918 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:44:06,918 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:44:06,922 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:07,179 [197] [DEBUG] [app] Starting request: urn:request:3a491ed0-dd4b-4bb5-9654-48342ea42982 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:44:07,179 [200] [DEBUG] [app] Starting request: urn:request:e870f4ce-76d8-4603-acde-1d3783760a0d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:44:07,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:07,181 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:07,183 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:07,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:44:07,184 [203] [DEBUG] [app] Starting request: urn:request:51c09b3d-da47-4eda-a181-599336b7ad21 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:44:07,184 [208] [DEBUG] [app] Starting request: urn:request:fa0cd0a5-b06c-466c-ae05-6b238501bb7d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:44:07,185 [203] [DEBUG] [app] Ending request: urn:request:51c09b3d-da47-4eda-a181-599336b7ad21 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:51c09b3d-da47-4eda-a181-599336b7ad21', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:44:07,185 [208] [DEBUG] [app] Ending request: urn:request:fa0cd0a5-b06c-466c-ae05-6b238501bb7d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fa0cd0a5-b06c-466c-ae05-6b238501bb7d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:44:07,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:44:07,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:44:07,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:07,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:07,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:07,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:07,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:07,190 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:07,190 [200] [DEBUG] [app] Starting request: urn:request:9bb74610-6416-4887-b491-0bf41ff63cf8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:44:07,191 [200] [DEBUG] [app] Ending request: urn:request:9bb74610-6416-4887-b491-0bf41ff63cf8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9bb74610-6416-4887-b491-0bf41ff63cf8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:44:07,191 [199] [DEBUG] [app] Starting request: urn:request:c1111d2a-8bda-42c0-a5f6-4b44d1fbb738 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:44:07,191 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.001) gunicorn-web stdout | 2023-08-08 06:44:07,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:07,191 [199] [DEBUG] [app] Ending request: urn:request:c1111d2a-8bda-42c0-a5f6-4b44d1fbb738 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c1111d2a-8bda-42c0-a5f6-4b44d1fbb738', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:44:07,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:44:07,192 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:44:07,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:07,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:44:07,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:44:07,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:44:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:44:07,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:37.197412+00:00 (in 29.999566 seconds) buildlogsarchiver stdout | 2023-08-08 06:44:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:37 UTC)" (scheduled at 2023-08-08 06:44:07.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:44:07,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 44, 7, 198255), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:44:07,201 [200] [DEBUG] [app] Ending request: urn:request:e870f4ce-76d8-4603-acde-1d3783760a0d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e870f4ce-76d8-4603-acde-1d3783760a0d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:44:07,201 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:07,201 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:44:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:44:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.023 47 0.023) gunicorn-web stdout | 2023-08-08 06:44:07,202 [197] [DEBUG] [app] Ending request: urn:request:3a491ed0-dd4b-4bb5-9654-48342ea42982 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3a491ed0-dd4b-4bb5-9654-48342ea42982', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:44:07,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:07,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:44:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:44:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) buildlogsarchiver stdout | 2023-08-08 06:44:07,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:44:07,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:44:07,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:44:07,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:44:07,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:44:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:44:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:19.417660+00:00 (in 9.999311 seconds) gcworker stdout | 2023-08-08 06:44:09,419 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:19 UTC)" (scheduled at 2023-08-08 06:44:09.417660+00:00) gcworker stdout | 2023-08-08 06:44:09,420 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267449419, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:44:09,429 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:44:09,429 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:44:09,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:44:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:44:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:19.187230+00:00 (in 9.393106 seconds) repositorygcworker stdout | 2023-08-08 06:44:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:09 UTC)" (scheduled at 2023-08-08 06:44:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:44:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:44:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 9, 794457), True, datetime.datetime(2023, 8, 8, 6, 44, 9, 794457), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:44:09,803 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:44:09,803 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:44:09,803 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:44:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:44:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:22.293757+00:00 (in 9.999380 seconds) chunkcleanupworker stdout | 2023-08-08 06:44:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:22 UTC)" (scheduled at 2023-08-08 06:44:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:44:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:44:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 12, 294777), True, datetime.datetime(2023, 8, 8, 6, 44, 12, 294777), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:44:12,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:44:12,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:44:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:44:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:44:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:54.284093+00:00 (in 41.186200 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:44:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:13 UTC)" (scheduled at 2023-08-08 06:44:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:44:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:44:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 13, 98270), True, datetime.datetime(2023, 8, 8, 6, 44, 13, 98270), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:44:13,105 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:44:13,106 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:44:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:44:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:44:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:25.371682+00:00 (in 9.999488 seconds) notificationworker stdout | 2023-08-08 06:44:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:25 UTC)" (scheduled at 2023-08-08 06:44:15.371682+00:00) notificationworker stdout | 2023-08-08 06:44:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:44:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 15, 372750), True, datetime.datetime(2023, 8, 8, 6, 44, 15, 372750), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:44:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:44:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:44:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:25 UTC)" executed successfully builder stdout | 2023-08-08 06:44:16,932 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:44:16,941 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:44:16,941 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:44:16,945 [55] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:44:19,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:44:19,187 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:04.187429+00:00 (in 44.999715 seconds) repositorygcworker stdout | 2023-08-08 06:44:19,187 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:49:19 UTC)" (scheduled at 2023-08-08 06:44:19.187230+00:00) repositorygcworker stdout | 2023-08-08 06:44:19,188 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 44, 19, 188005), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:44:19,195 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 44, 19, 188005), True, datetime.datetime(2023, 8, 8, 6, 44, 19, 188005), 0, 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:44:19,197 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 44, 19, 188005), True, datetime.datetime(2023, 8, 8, 6, 44, 19, 188005), 0, 'repositorygc/%', False, datetime.datetime(2023, 8, 8, 6, 44, 19, 188005), 'repositorygc/%']) repositorygcworker stdout | 2023-08-08 06:44:19,198 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:44:19,198 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:49:19 UTC)" executed successfully gcworker stdout | 2023-08-08 06:44:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:44:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:29.417660+00:00 (in 9.999530 seconds) gcworker stdout | 2023-08-08 06:44:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:29 UTC)" (scheduled at 2023-08-08 06:44:19.417660+00:00) gcworker stdout | 2023-08-08 06:44:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267459418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:44:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:44:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:44:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:44:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:44:19,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:26.890741+00:00 (in 7.202906 seconds) securityworker stdout | 2023-08-08 06:44:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:49 UTC)" (scheduled at 2023-08-08 06:44:19.687363+00:00) securityworker stdout | 2023-08-08 06:44:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:44:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:44:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:44:19,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:44:19,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:44:19,699 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:44:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:19,701 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 39, 19, 691834), 1, 2]) securityworker stdout | 2023-08-08 06:44:19,702 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:19,702 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 39, 19, 691834), 1, 2]) securityworker stdout | 2023-08-08 06:44:19,704 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:19,704 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:19,704 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:44:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:49 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:44:20,094 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:44:21,081 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:44:22,179 [197] [DEBUG] [app] Starting request: urn:request:af356ac2-7a17-486d-9a1b-ad03a3841c88 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:44:22,180 [199] [DEBUG] [app] Starting request: urn:request:d622d3e4-1f16-46c8-a450-20643df1828b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:44:22,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:22,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:22,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:22,184 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:44:22,185 [209] [DEBUG] [app] Starting request: urn:request:273e6ff9-7b88-4abd-b86e-2c504eaf5e2e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:44:22,185 [208] [DEBUG] [app] Starting request: urn:request:461c1ef9-b81d-4080-915c-b4d1d548d98e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:44:22,186 [209] [DEBUG] [app] Ending request: urn:request:273e6ff9-7b88-4abd-b86e-2c504eaf5e2e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:273e6ff9-7b88-4abd-b86e-2c504eaf5e2e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:44:22,186 [208] [DEBUG] [app] Ending request: urn:request:461c1ef9-b81d-4080-915c-b4d1d548d98e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:461c1ef9-b81d-4080-915c-b4d1d548d98e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:44:22,186 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:44:22,186 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:44:22,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:44:22,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:22,188 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:22,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:22,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:22,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:22,192 [197] [DEBUG] [app] Starting request: urn:request:6cc575f9-f270-46e1-aa80-056311ab6b97 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:44:22,192 [200] [DEBUG] [app] Starting request: urn:request:bc7e473f-5998-403b-9fac-f5ad1046859b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:44:22,192 [197] [DEBUG] [app] Ending request: urn:request:6cc575f9-f270-46e1-aa80-056311ab6b97 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6cc575f9-f270-46e1-aa80-056311ab6b97', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:44:22,192 [200] [DEBUG] [app] Ending request: urn:request:bc7e473f-5998-403b-9fac-f5ad1046859b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bc7e473f-5998-403b-9fac-f5ad1046859b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:44:22,192 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:44:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:44:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:44:22,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:22,193 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:44:22,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:44:22,198 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:44:22,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:44:22,204 [197] [DEBUG] [app] Ending request: urn:request:af356ac2-7a17-486d-9a1b-ad03a3841c88 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:af356ac2-7a17-486d-9a1b-ad03a3841c88', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:44:22,204 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:22,204 [199] [DEBUG] [app] Ending request: urn:request:d622d3e4-1f16-46c8-a450-20643df1828b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d622d3e4-1f16-46c8-a450-20643df1828b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:44:22,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:22,204 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:44:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:44:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:44:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.026) gunicorn-web stdout | 2023-08-08 06:44:22,204 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:44:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" manifestbackfillworker stdout | 2023-08-08 06:44:22,250 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} chunkcleanupworker stdout | 2023-08-08 06:44:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:44:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:28.874906+00:00 (in 6.580619 seconds) chunkcleanupworker stdout | 2023-08-08 06:44:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:32 UTC)" (scheduled at 2023-08-08 06:44:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:44:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:44:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 22, 294743), True, datetime.datetime(2023, 8, 8, 6, 44, 22, 294743), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:44:22,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:44:22,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:44:22,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:32 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:44:22,787 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} notificationworker stdout | 2023-08-08 06:44:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:44:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:35.371682+00:00 (in 9.999532 seconds) notificationworker stdout | 2023-08-08 06:44:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:35 UTC)" (scheduled at 2023-08-08 06:44:25.371682+00:00) notificationworker stdout | 2023-08-08 06:44:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:44:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 25, 372596), True, datetime.datetime(2023, 8, 8, 6, 44, 25, 372596), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:44:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:44:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:44:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:35 UTC)" executed successfully gcworker stdout | 2023-08-08 06:44:25,470 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} exportactionlogsworker stdout | 2023-08-08 06:44:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:44:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:47.398551+00:00 (in 21.883699 seconds) exportactionlogsworker stdout | 2023-08-08 06:44:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:25 UTC)" (scheduled at 2023-08-08 06:44:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:44:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:44:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:25 UTC)" executed successfully gunicorn-secscan stdout | 2023-08-08 06:44:25,656 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:44:25,659 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:44:25,661 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} securityworker stdout | 2023-08-08 06:44:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:44:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:49.687363+00:00 (in 22.796139 seconds) securityworker stdout | 2023-08-08 06:44:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:56 UTC)" (scheduled at 2023-08-08 06:44:26.890741+00:00) securityworker stdout | 2023-08-08 06:44:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:44:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:44:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:44:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:44:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:44:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:44:26,893 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:44:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:44:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:44:26,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:44:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:26,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 39, 26, 896343), 1, 2]) securityworker stdout | 2023-08-08 06:44:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 39, 26, 896343), 1, 2]) securityworker stdout | 2023-08-08 06:44:26,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:26,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:26,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:44:26,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:44:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:44:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:44:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:44:26,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:44:26,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:44:26,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:44:56 UTC)" executed successfully builder stdout | 2023-08-08 06:44:26,950 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:44:26,959 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:44:26,959 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:44:26,963 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:44:27,301 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:44:27,617 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:44:27,679 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:44:28,041 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:44:28,388 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:44:28,523 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:44:28,527 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:44:28,539 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:44:28,671 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} chunkcleanupworker stdout | 2023-08-08 06:44:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:44:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:32.293757+00:00 (in 3.418363 seconds) chunkcleanupworker stdout | 2023-08-08 06:44:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:28 UTC)" (scheduled at 2023-08-08 06:44:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:44:28,875 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:44:28,875 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:28 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 06:44:29,158 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} queuecleanupworker stdout | 2023-08-08 06:44:29,390 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:44:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:44:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:39.417660+00:00 (in 9.999499 seconds) gcworker stdout | 2023-08-08 06:44:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:39 UTC)" (scheduled at 2023-08-08 06:44:29.417660+00:00) gcworker stdout | 2023-08-08 06:44:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267469418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:44:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:44:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:44:29,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:39 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:44:30,761 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:44:30,956 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:44:31,951 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:44:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:44:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:42.293757+00:00 (in 9.999440 seconds) chunkcleanupworker stdout | 2023-08-08 06:44:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:42 UTC)" (scheduled at 2023-08-08 06:44:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:44:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:44:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 32, 294701), True, datetime.datetime(2023, 8, 8, 6, 44, 32, 294701), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:44:32,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:44:32,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:44:32,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:44:32,787 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:44:32,964 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:44:33,250 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:44:33,913 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:44:34,043 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:44:34,046 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:44:34,051 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:44:34,054 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:44:34,060 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:44:34,063 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:44:34,067 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:44:34,072 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:44:34,078 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} namespacegcworker stdout | 2023-08-08 06:44:34,207 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:44:34,691 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:44:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:44:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:45.371682+00:00 (in 9.999500 seconds) notificationworker stdout | 2023-08-08 06:44:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:45 UTC)" (scheduled at 2023-08-08 06:44:35.371682+00:00) notificationworker stdout | 2023-08-08 06:44:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:44:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 35, 372634), True, datetime.datetime(2023, 8, 8, 6, 44, 35, 372634), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:44:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:44:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:44:35,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:44:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:44:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:39.579576+00:00 (in 2.999195 seconds) namespacegcworker stdout | 2023-08-08 06:44:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:36 UTC)" (scheduled at 2023-08-08 06:44:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:44:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:44:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:36 UTC)" executed successfully builder stdout | 2023-08-08 06:44:36,973 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:44:36,982 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:44:36,982 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:44:36,986 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:37,179 [198] [DEBUG] [app] Starting request: urn:request:cab33aff-614a-45a7-8848-110b348d9106 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:44:37,179 [197] [DEBUG] [app] Starting request: urn:request:9fa8535f-6292-44cf-9d4a-187b40b224c8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:44:37,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:37,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:37,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:37,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:44:37,185 [210] [DEBUG] [app] Starting request: urn:request:0f34b9b5-16b9-4cad-a921-68fae6485578 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:44:37,185 [207] [DEBUG] [app] Starting request: urn:request:f7ee1e55-8afb-4e4c-870b-edae82a0dddf (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:44:37,185 [210] [DEBUG] [app] Ending request: urn:request:0f34b9b5-16b9-4cad-a921-68fae6485578 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0f34b9b5-16b9-4cad-a921-68fae6485578', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:44:37,185 [207] [DEBUG] [app] Ending request: urn:request:f7ee1e55-8afb-4e4c-870b-edae82a0dddf (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f7ee1e55-8afb-4e4c-870b-edae82a0dddf', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:44:37,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:44:37,186 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:44:37,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:37,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:37,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:37,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:37,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:37,191 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:37,191 [198] [DEBUG] [app] Starting request: urn:request:71a93d41-8b46-4d71-a1fd-e0606267faab (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:44:37,192 [198] [DEBUG] [app] Ending request: urn:request:71a93d41-8b46-4d71-a1fd-e0606267faab (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:71a93d41-8b46-4d71-a1fd-e0606267faab', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:44:37,192 [200] [DEBUG] [app] Starting request: urn:request:86608044-b61d-4dcf-9b19-4c46dfee0f3b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:44:37,192 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:44:37,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:37,192 [200] [DEBUG] [app] Ending request: urn:request:86608044-b61d-4dcf-9b19-4c46dfee0f3b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:86608044-b61d-4dcf-9b19-4c46dfee0f3b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:44:37,193 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:44:37,193 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:44:37,193 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:37,193 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:44:37,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:44:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:44:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:07.197412+00:00 (in 29.999567 seconds) buildlogsarchiver stdout | 2023-08-08 06:44:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:07 UTC)" (scheduled at 2023-08-08 06:44:37.197412+00:00) gunicorn-web stdout | 2023-08-08 06:44:37,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:44:37,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 44, 37, 198256), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:44:37,202 [198] [DEBUG] [app] Ending request: urn:request:cab33aff-614a-45a7-8848-110b348d9106 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cab33aff-614a-45a7-8848-110b348d9106', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:44:37,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:37,203 [197] [DEBUG] [app] Ending request: urn:request:9fa8535f-6292-44cf-9d4a-187b40b224c8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9fa8535f-6292-44cf-9d4a-187b40b224c8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:44:37,203 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:37,203 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:44:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:44:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:44:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) gunicorn-web stdout | 2023-08-08 06:44:37,203 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:44:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:44:37,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:44:37,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:44:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:44:37,209 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:44:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:44:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:44:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:49.417660+00:00 (in 9.999492 seconds) gcworker stdout | 2023-08-08 06:44:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:49 UTC)" (scheduled at 2023-08-08 06:44:39.417660+00:00) gcworker stdout | 2023-08-08 06:44:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267479418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:44:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:44:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:44:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:49 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:44:39,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:44:39,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:54.393627+00:00 (in 14.813315 seconds) namespacegcworker stdout | 2023-08-08 06:44:39,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:49:39 UTC)" (scheduled at 2023-08-08 06:44:39.579576+00:00) namespacegcworker stdout | 2023-08-08 06:44:39,581 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 44, 39, 580758), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:44:39,587 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 44, 39, 580758), True, datetime.datetime(2023, 8, 8, 6, 44, 39, 580758), 0, 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:44:39,589 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 44, 39, 580758), True, datetime.datetime(2023, 8, 8, 6, 44, 39, 580758), 0, 'namespacegc/%', False, datetime.datetime(2023, 8, 8, 6, 44, 39, 580758), 'namespacegc/%']) namespacegcworker stdout | 2023-08-08 06:44:39,590 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:44:39,590 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:49:39 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:44:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:44:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:46.874674+00:00 (in 4.580408 seconds) chunkcleanupworker stdout | 2023-08-08 06:44:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:52 UTC)" (scheduled at 2023-08-08 06:44:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:44:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:44:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 42, 294642), True, datetime.datetime(2023, 8, 8, 6, 44, 42, 294642), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:44:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:44:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:44:42,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:44:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:44:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:55.371682+00:00 (in 9.999467 seconds) notificationworker stdout | 2023-08-08 06:44:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:55 UTC)" (scheduled at 2023-08-08 06:44:45.371682+00:00) notificationworker stdout | 2023-08-08 06:44:45,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:44:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 45, 372771), True, datetime.datetime(2023, 8, 8, 6, 44, 45, 372771), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:44:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:44:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:44:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:55 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:44:46,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:44:46,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:52.293757+00:00 (in 5.418515 seconds) chunkcleanupworker stdout | 2023-08-08 06:44:46,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:49:46 UTC)" (scheduled at 2023-08-08 06:44:46.874674+00:00) chunkcleanupworker stdout | 2023-08-08 06:44:46,876 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2023, 8, 8, 6, 44, 46, 875781), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:44:46,883 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 44, 46, 875781), True, datetime.datetime(2023, 8, 8, 6, 44, 46, 875781), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:44:46,885 [57] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2023, 8, 8, 6, 44, 46, 875781), True, datetime.datetime(2023, 8, 8, 6, 44, 46, 875781), 0, 'chunk_cleanup/%', False, datetime.datetime(2023, 8, 8, 6, 44, 46, 875781), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2023-08-08 06:44:46,886 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:44:46,886 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2023-08-08 06:49:46 UTC)" executed successfully builder stdout | 2023-08-08 06:44:46,997 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:44:47,006 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:44:47,006 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:44:47,010 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:44:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:44:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:25.514341+00:00 (in 38.115311 seconds) exportactionlogsworker stdout | 2023-08-08 06:44:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:47 UTC)" (scheduled at 2023-08-08 06:44:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:44:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:44:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 47, 399368), True, datetime.datetime(2023, 8, 8, 6, 44, 47, 399368), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:44:47,406 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:44:47,406 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:44:47,406 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:44:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:44:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:59.417660+00:00 (in 9.999464 seconds) gcworker stdout | 2023-08-08 06:44:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:59 UTC)" (scheduled at 2023-08-08 06:44:49.417660+00:00) gcworker stdout | 2023-08-08 06:44:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267489418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:44:49,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:44:49,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:44:49,429 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:44:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:44:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:44:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:44:56.890741+00:00 (in 7.202839 seconds) securityworker stdout | 2023-08-08 06:44:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:19 UTC)" (scheduled at 2023-08-08 06:44:49.687363+00:00) securityworker stdout | 2023-08-08 06:44:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:44:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:44:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:44:49,692 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:44:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:44:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:44:50,131 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:44:51,104 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:44:52,178 [197] [DEBUG] [app] Starting request: urn:request:6030feba-ed1b-46be-a1ba-35a642e309d0 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:44:52,179 [200] [DEBUG] [app] Starting request: urn:request:ed080ee4-7602-4295-b8b6-678640db61f1 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:44:52,180 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:44:52,184 [207] [DEBUG] [app] Starting request: urn:request:80019b2a-9b5b-4d59-869f-d26c3e3a26fe (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:44:52,185 [208] [DEBUG] [app] Starting request: urn:request:70786f5e-5eac-4668-8b8c-ce143966ca94 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:44:52,185 [207] [DEBUG] [app] Ending request: urn:request:80019b2a-9b5b-4d59-869f-d26c3e3a26fe (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:80019b2a-9b5b-4d59-869f-d26c3e3a26fe', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:44:52,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:44:52,185 [208] [DEBUG] [app] Ending request: urn:request:70786f5e-5eac-4668-8b8c-ce143966ca94 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:70786f5e-5eac-4668-8b8c-ce143966ca94', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:44:52,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:44:52,185 [208] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:44:52,185 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:44:52,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:52,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:44:52,190 [199] [DEBUG] [app] Starting request: urn:request:2258585b-ec99-4777-a7a1-7f9afdba6083 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:44:52,190 [199] [DEBUG] [app] Ending request: urn:request:2258585b-ec99-4777-a7a1-7f9afdba6083 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2258585b-ec99-4777-a7a1-7f9afdba6083', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:44:52,190 [198] [DEBUG] [app] Starting request: urn:request:fbf2e828-cc74-4539-a4eb-e57c19b0e009 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:44:52,190 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:44:52,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:52,191 [198] [DEBUG] [app] Ending request: urn:request:fbf2e828-cc74-4539-a4eb-e57c19b0e009 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fbf2e828-cc74-4539-a4eb-e57c19b0e009', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:44:52,191 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:44:52,191 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:44:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:44:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:44:52,191 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:44:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:44:52,196 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:44:52,196 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:44:52,201 [200] [DEBUG] [app] Ending request: urn:request:ed080ee4-7602-4295-b8b6-678640db61f1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ed080ee4-7602-4295-b8b6-678640db61f1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:44:52,201 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:44:52,202 [197] [DEBUG] [app] Ending request: urn:request:6030feba-ed1b-46be-a1ba-35a642e309d0 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6030feba-ed1b-46be-a1ba-35a642e309d0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:44:52,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:44:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:44:52,202 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:44:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:44:52,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:44:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:44:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.024) manifestbackfillworker stdout | 2023-08-08 06:44:52,288 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} chunkcleanupworker stdout | 2023-08-08 06:44:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:44:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:02.293757+00:00 (in 9.999457 seconds) chunkcleanupworker stdout | 2023-08-08 06:44:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:02 UTC)" (scheduled at 2023-08-08 06:44:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:44:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:44:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 52, 294768), True, datetime.datetime(2023, 8, 8, 6, 44, 52, 294768), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:44:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:44:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:44:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:02 UTC)" executed successfully storagereplication stdout | 2023-08-08 06:44:52,816 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:44:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:44:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:13.097375+00:00 (in 18.812789 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:44:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:54 UTC)" (scheduled at 2023-08-08 06:44:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:44:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:44:54,284 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:44:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:44:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:36.579841+00:00 (in 42.185694 seconds) namespacegcworker stdout | 2023-08-08 06:44:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:54 UTC)" (scheduled at 2023-08-08 06:44:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:44:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:44:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 54, 394698), True, datetime.datetime(2023, 8, 8, 6, 44, 54, 394698), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:44:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:44:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:44:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:45:54 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:44:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:44:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:02.178530+00:00 (in 6.806265 seconds) notificationworker stdout | 2023-08-08 06:44:55,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:05 UTC)" (scheduled at 2023-08-08 06:44:55.371682+00:00) notificationworker stdout | 2023-08-08 06:44:55,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:44:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 44, 55, 372810), True, datetime.datetime(2023, 8, 8, 6, 44, 55, 372810), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:44:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:44:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:44:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:05 UTC)" executed successfully gcworker stdout | 2023-08-08 06:44:55,494 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:44:55,665 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:44:55,668 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:44:55,671 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} securityworker stdout | 2023-08-08 06:44:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:44:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:19.687363+00:00 (in 22.796158 seconds) securityworker stdout | 2023-08-08 06:44:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:26 UTC)" (scheduled at 2023-08-08 06:44:56.890741+00:00) securityworker stdout | 2023-08-08 06:44:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:44:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:44:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:44:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:44:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:44:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:44:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:44:56,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:44:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:44:56,903 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:44:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:56,905 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 39, 56, 896172), 1, 2]) securityworker stdout | 2023-08-08 06:44:56,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:56,907 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 39, 56, 896172), 1, 2]) securityworker stdout | 2023-08-08 06:44:56,908 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:44:56,908 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:44:56,908 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:44:56,908 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:44:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:44:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:44:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:44:56,909 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:44:56,909 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:44:56,909 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:26 UTC)" executed successfully builder stdout | 2023-08-08 06:44:57,020 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:44:57,029 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:44:57,029 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:44:57,033 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:44:57,326 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:44:57,650 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:44:57,716 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:44:58,051 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:44:58,400 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:44:58,539 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:44:58,542 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:44:58,551 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:44:58,706 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} repositoryactioncounter stdout | 2023-08-08 06:44:59,166 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gcworker stdout | 2023-08-08 06:44:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:44:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:09.417660+00:00 (in 9.999458 seconds) gcworker stdout | 2023-08-08 06:44:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:09 UTC)" (scheduled at 2023-08-08 06:44:59.417660+00:00) gcworker stdout | 2023-08-08 06:44:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267499418, 1209600, True, 3, 500, 1, 0]) queuecleanupworker stdout | 2023-08-08 06:44:59,426 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} gcworker stdout | 2023-08-08 06:44:59,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:44:59,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:44:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:09 UTC)" executed successfully buildlogsarchiver stdout | 2023-08-08 06:45:00,793 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:45:00,993 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:45:01,986 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} notificationworker stdout | 2023-08-08 06:45:02,178 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:45:02,179 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:05.371682+00:00 (in 3.192597 seconds) notificationworker stdout | 2023-08-08 06:45:02,179 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:02 UTC)" (scheduled at 2023-08-08 06:45:02.178530+00:00) notificationworker stdout | 2023-08-08 06:45:02,179 [71] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2023-08-08 06:45:02,179 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:02 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:45:02,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:45:02,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:12.293757+00:00 (in 9.999475 seconds) chunkcleanupworker stdout | 2023-08-08 06:45:02,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:12 UTC)" (scheduled at 2023-08-08 06:45:02.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:45:02,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:45:02,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 2, 294906), True, datetime.datetime(2023, 8, 8, 6, 45, 2, 294906), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:45:02,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:45:02,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:45:02,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:12 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:45:02,825 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:45:03,001 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:45:03,267 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:45:03,944 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:45:04,055 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:45:04,058 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:45:04,061 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:45:04,065 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:45:04,076 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:45:04,079 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:45:04,084 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:45:04,088 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:45:04,093 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} repositorygcworker stdout | 2023-08-08 06:45:04,187 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:45:04,188 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:09.793658+00:00 (in 5.605768 seconds) repositorygcworker stdout | 2023-08-08 06:45:04,188 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:04 UTC)" (scheduled at 2023-08-08 06:45:04.187429+00:00) repositorygcworker stdout | 2023-08-08 06:45:04,188 [75] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2023-08-08 06:45:04,188 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:04 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:45:04,241 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:45:04,726 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:45:05,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:45:05,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:15.371682+00:00 (in 9.999484 seconds) notificationworker stdout | 2023-08-08 06:45:05,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:15 UTC)" (scheduled at 2023-08-08 06:45:05.371682+00:00) notificationworker stdout | 2023-08-08 06:45:05,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:45:05,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 5, 372680), True, datetime.datetime(2023, 8, 8, 6, 45, 5, 372680), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:45:05,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:45:05,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:45:05,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:15 UTC)" executed successfully builder stdout | 2023-08-08 06:45:07,040 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:45:07,049 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:45:07,049 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:45:07,053 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:07,179 [199] [DEBUG] [app] Starting request: urn:request:33486b23-6d7b-44c5-93d0-292348e8f2db (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:45:07,179 [197] [DEBUG] [app] Starting request: urn:request:b69d12aa-4d76-4993-a52e-0d47ee6ad4ff (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:45:07,181 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:07,182 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:07,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:07,185 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:45:07,185 [210] [DEBUG] [app] Starting request: urn:request:7c401bc0-da3f-4f49-980f-3de328986e6f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:45:07,186 [210] [DEBUG] [app] Ending request: urn:request:7c401bc0-da3f-4f49-980f-3de328986e6f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7c401bc0-da3f-4f49-980f-3de328986e6f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:45:07,186 [203] [DEBUG] [app] Starting request: urn:request:fde2c188-6893-40b9-8cdb-ac4ff25d8489 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:45:07,186 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-registry stdout | 2023-08-08 06:45:07,187 [203] [DEBUG] [app] Ending request: urn:request:fde2c188-6893-40b9-8cdb-ac4ff25d8489 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fde2c188-6893-40b9-8cdb-ac4ff25d8489', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:45:07,187 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:45:07,187 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:07 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:45:07,187 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:07,188 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:07,190 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:07,191 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:07,192 [197] [DEBUG] [app] Starting request: urn:request:f22c06a2-9508-4b5d-8610-18e91b84bddc (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:45:07,193 [197] [DEBUG] [app] Ending request: urn:request:f22c06a2-9508-4b5d-8610-18e91b84bddc (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f22c06a2-9508-4b5d-8610-18e91b84bddc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:45:07,193 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:45:07,193 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:07,194 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:45:07,194 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:07,195 [198] [DEBUG] [app] Starting request: urn:request:f4bd1994-1e08-4e40-bd56-64509ab024f3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:45:07,196 [198] [DEBUG] [app] Ending request: urn:request:f4bd1994-1e08-4e40-bd56-64509ab024f3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:f4bd1994-1e08-4e40-bd56-64509ab024f3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:07 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:45:07,196 [198] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:07 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:45:07,196 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:07,197 [197] [INFO] [data.database] Connection pooling disabled for postgresql buildlogsarchiver stdout | 2023-08-08 06:45:07,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:45:07,197 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:37.197412+00:00 (in 29.999570 seconds) buildlogsarchiver stdout | 2023-08-08 06:45:07,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:37 UTC)" (scheduled at 2023-08-08 06:45:07.197412+00:00) gunicorn-web stdout | 2023-08-08 06:45:07,199 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:45:07,199 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 45, 7, 198470), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:45:07,202 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:45:07,204 [199] [DEBUG] [app] Ending request: urn:request:33486b23-6d7b-44c5-93d0-292348e8f2db (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:33486b23-6d7b-44c5-93d0-292348e8f2db', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:45:07,204 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:07,205 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:45:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:45:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.026 47 0.026) buildlogsarchiver stdout | 2023-08-08 06:45:07,206 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:45:07,207 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 gunicorn-web stdout | 2023-08-08 06:45:07,207 [197] [DEBUG] [app] Ending request: urn:request:b69d12aa-4d76-4993-a52e-0d47ee6ad4ff (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b69d12aa-4d76-4993-a52e-0d47ee6ad4ff', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:45:07,207 [197] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:45:07 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.029 47 0.030) gunicorn-web stdout | 2023-08-08 06:45:07,208 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:45:07 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:45:07,208 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:45:07,213 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:45:07,213 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:37 UTC)" executed successfully gcworker stdout | 2023-08-08 06:45:09,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:45:09,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:19.417660+00:00 (in 9.999506 seconds) gcworker stdout | 2023-08-08 06:45:09,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:19 UTC)" (scheduled at 2023-08-08 06:45:09.417660+00:00) gcworker stdout | 2023-08-08 06:45:09,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267509418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:45:09,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:45:09,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:45:09,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:19 UTC)" executed successfully repositorygcworker stdout | 2023-08-08 06:45:09,794 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2023-08-08 06:45:09,794 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:04.187429+00:00 (in 54.393306 seconds) repositorygcworker stdout | 2023-08-08 06:45:09,794 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:09 UTC)" (scheduled at 2023-08-08 06:45:09.793658+00:00) repositorygcworker stdout | 2023-08-08 06:45:09,794 [75] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2023-08-08 06:45:09,795 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 9, 794458), True, datetime.datetime(2023, 8, 8, 6, 45, 9, 794458), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2023-08-08 06:45:09,802 [75] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2023-08-08 06:45:09,802 [75] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2023-08-08 06:45:09,802 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:09 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:45:12,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:45:12,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:22.293757+00:00 (in 9.999478 seconds) chunkcleanupworker stdout | 2023-08-08 06:45:12,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:22 UTC)" (scheduled at 2023-08-08 06:45:12.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:45:12,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:45:12,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 12, 294662), True, datetime.datetime(2023, 8, 8, 6, 45, 12, 294662), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:45:12,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:45:12,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:45:12,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:22 UTC)" executed successfully securityscanningnotificationworker stdout | 2023-08-08 06:45:13,097 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:45:13,098 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:54.284093+00:00 (in 41.186217 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:45:13,098 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:13 UTC)" (scheduled at 2023-08-08 06:45:13.097375+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:45:13,098 [76] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2023-08-08 06:45:13,099 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 13, 98248), True, datetime.datetime(2023, 8, 8, 6, 45, 13, 98248), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2023-08-08 06:45:13,105 [76] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2023-08-08 06:45:13,105 [76] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2023-08-08 06:45:13,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:13 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:45:15,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:45:15,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:25.371682+00:00 (in 9.999476 seconds) notificationworker stdout | 2023-08-08 06:45:15,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:25 UTC)" (scheduled at 2023-08-08 06:45:15.371682+00:00) notificationworker stdout | 2023-08-08 06:45:15,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:45:15,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 15, 372711), True, datetime.datetime(2023, 8, 8, 6, 45, 15, 372711), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:45:15,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:45:15,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:45:15,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:25 UTC)" executed successfully builder stdout | 2023-08-08 06:45:17,063 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:45:17,072 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:45:17,072 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:45:17,076 [55] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:45:19,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:45:19,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:29.417660+00:00 (in 9.999504 seconds) gcworker stdout | 2023-08-08 06:45:19,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:29 UTC)" (scheduled at 2023-08-08 06:45:19.417660+00:00) gcworker stdout | 2023-08-08 06:45:19,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267519418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:45:19,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:45:19,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:45:19,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:29 UTC)" executed successfully securityworker stdout | 2023-08-08 06:45:19,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:45:19,687 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:26.890741+00:00 (in 7.202920 seconds) securityworker stdout | 2023-08-08 06:45:19,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:49 UTC)" (scheduled at 2023-08-08 06:45:19.687363+00:00) securityworker stdout | 2023-08-08 06:45:19,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:45:19,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:45:19,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:45:19,691 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:45:19,697 [77] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:45:19,698 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:45:19,700 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:19,700 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:19,700 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 40, 19, 691488), 1, 2]) securityworker stdout | 2023-08-08 06:45:19,703 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:19,703 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:19,703 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 40, 19, 691488), 1, 2]) securityworker stdout | 2023-08-08 06:45:19,705 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:19,705 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:19,705 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:45:19,705 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:49 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:45:20,168 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:45:21,112 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:45:22,178 [199] [DEBUG] [app] Starting request: urn:request:7cf50d19-7681-4705-9937-df503303d883 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:45:22,179 [198] [DEBUG] [app] Starting request: urn:request:858ef818-4228-4257-8da3-c06b1efb2f17 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:45:22,180 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:22,181 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:22,183 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:22,183 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:45:22,184 [203] [DEBUG] [app] Starting request: urn:request:c5cba3d0-fede-4a54-b021-d08f69bba33b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:45:22,184 [209] [DEBUG] [app] Starting request: urn:request:1c15f4e9-f424-4218-8cb9-25d93be4bbe1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:45:22,185 [203] [DEBUG] [app] Ending request: urn:request:c5cba3d0-fede-4a54-b021-d08f69bba33b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c5cba3d0-fede-4a54-b021-d08f69bba33b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:45:22,185 [209] [DEBUG] [app] Ending request: urn:request:1c15f4e9-f424-4218-8cb9-25d93be4bbe1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1c15f4e9-f424-4218-8cb9-25d93be4bbe1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:45:22,185 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.001) gunicorn-registry stdout | 2023-08-08 06:45:22,185 [209] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:45:22,185 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:22 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:45:22,186 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:22,187 [199] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:22,187 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:22,189 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:22,190 [199] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:22,190 [197] [DEBUG] [app] Starting request: urn:request:ce9c608f-a729-499c-9b4e-07377a3a106c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:45:22,191 [197] [DEBUG] [app] Ending request: urn:request:ce9c608f-a729-499c-9b4e-07377a3a106c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ce9c608f-a729-499c-9b4e-07377a3a106c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:45:22,191 [200] [DEBUG] [app] Starting request: urn:request:5fc5291d-d3ad-49ec-a3e2-ba8475ae2eae (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:45:22,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:45:22,191 [200] [DEBUG] [app] Ending request: urn:request:5fc5291d-d3ad-49ec-a3e2-ba8475ae2eae (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5fc5291d-d3ad-49ec-a3e2-ba8475ae2eae', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:45:22,192 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:22,192 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:22 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:22 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2023-08-08 06:45:22,192 [199] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:22,192 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:45:22,192 [199] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:45:22,197 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:45:22,197 [199] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:45:22,202 [199] [DEBUG] [app] Ending request: urn:request:7cf50d19-7681-4705-9937-df503303d883 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7cf50d19-7681-4705-9937-df503303d883', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:45:22,202 [198] [DEBUG] [app] Ending request: urn:request:858ef818-4228-4257-8da3-c06b1efb2f17 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:858ef818-4228-4257-8da3-c06b1efb2f17', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:45:22,202 [199] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:22,202 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:22,202 [199] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:45:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" gunicorn-web stdout | 2023-08-08 06:45:22,202 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:45:22 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:45:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.025 47 0.025) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:45:22 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:45:22,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:45:22,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:28.874906+00:00 (in 6.580597 seconds) chunkcleanupworker stdout | 2023-08-08 06:45:22,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:32 UTC)" (scheduled at 2023-08-08 06:45:22.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:45:22,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:45:22,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 22, 294771), True, datetime.datetime(2023, 8, 8, 6, 45, 22, 294771), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:45:22,303 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:45:22,303 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:45:22,303 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:32 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:45:22,305 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:45:22,857 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} notificationworker stdout | 2023-08-08 06:45:25,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:45:25,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:35.371682+00:00 (in 9.999488 seconds) notificationworker stdout | 2023-08-08 06:45:25,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:35 UTC)" (scheduled at 2023-08-08 06:45:25.371682+00:00) notificationworker stdout | 2023-08-08 06:45:25,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:45:25,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 25, 372642), True, datetime.datetime(2023, 8, 8, 6, 45, 25, 372642), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:45:25,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:45:25,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:45:25,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:35 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:45:25,514 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:45:25,515 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:47.398551+00:00 (in 21.883710 seconds) exportactionlogsworker stdout | 2023-08-08 06:45:25,515 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:25 UTC)" (scheduled at 2023-08-08 06:45:25.514341+00:00) exportactionlogsworker stdout | 2023-08-08 06:45:25,515 [60] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2023-08-08 06:45:25,515 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:25 UTC)" executed successfully gcworker stdout | 2023-08-08 06:45:25,530 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:45:25,674 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:45:25,677 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:45:25,680 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} securityworker stdout | 2023-08-08 06:45:26,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:45:26,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:49.687363+00:00 (in 22.796146 seconds) securityworker stdout | 2023-08-08 06:45:26,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:56 UTC)" (scheduled at 2023-08-08 06:45:26.890741+00:00) securityworker stdout | 2023-08-08 06:45:26,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:45:26,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:45:26,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:45:26,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:45:26,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:45:26,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:45:26,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:45:26,895 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:45:26,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:45:26,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:45:26,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:26,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:26,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 40, 26, 896332), 1, 2]) securityworker stdout | 2023-08-08 06:45:26,906 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:26,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:26,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 40, 26, 896332), 1, 2]) securityworker stdout | 2023-08-08 06:45:26,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:26,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:26,907 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:45:26,907 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:45:26,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:45:26,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:45:26,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:45:26,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:45:26,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:45:26,908 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:45:56 UTC)" executed successfully builder stdout | 2023-08-08 06:45:27,086 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:45:27,095 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:45:27,095 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:45:27,100 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:45:27,348 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:45:27,688 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:45:27,729 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:45:28,060 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:45:28,413 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:45:28,555 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:45:28,558 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:45:28,566 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:45:28,743 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} chunkcleanupworker stdout | 2023-08-08 06:45:28,875 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:45:28,875 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:32.293757+00:00 (in 3.418309 seconds) chunkcleanupworker stdout | 2023-08-08 06:45:28,875 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:28 UTC)" (scheduled at 2023-08-08 06:45:28.874906+00:00) chunkcleanupworker stdout | 2023-08-08 06:45:28,876 [57] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2023-08-08 06:45:28,876 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:28 UTC)" executed successfully repositoryactioncounter stdout | 2023-08-08 06:45:29,194 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gcworker stdout | 2023-08-08 06:45:29,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:45:29,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:39.417660+00:00 (in 9.999493 seconds) gcworker stdout | 2023-08-08 06:45:29,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:39 UTC)" (scheduled at 2023-08-08 06:45:29.417660+00:00) gcworker stdout | 2023-08-08 06:45:29,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267529418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:45:29,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:45:29,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:45:29,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:39 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:45:29,453 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'} buildlogsarchiver stdout | 2023-08-08 06:45:30,831 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'buildlogsarchiver.py', 'pid': '56'} builder stdout | 2023-08-08 06:45:31,027 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'builder.py', 'pid': '55'} securityworker stdout | 2023-08-08 06:45:32,002 [77] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityworker.py', 'pid': '77'} chunkcleanupworker stdout | 2023-08-08 06:45:32,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:45:32,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:42.293757+00:00 (in 9.999463 seconds) chunkcleanupworker stdout | 2023-08-08 06:45:32,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:42 UTC)" (scheduled at 2023-08-08 06:45:32.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:45:32,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:45:32,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 32, 294830), True, datetime.datetime(2023, 8, 8, 6, 45, 32, 294830), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:45:32,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:45:32,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:45:32,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:42 UTC)" executed successfully exportactionlogsworker stdout | 2023-08-08 06:45:32,862 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'exportactionlogsworker.py', 'pid': '60'} servicekey stdout | 2023-08-08 06:45:33,028 [78] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'servicekeyworker.py', 'pid': '78'} globalpromstats stdout | 2023-08-08 06:45:33,296 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'globalpromstats.py', 'pid': '62'} securityscanningnotificationworker stdout | 2023-08-08 06:45:33,971 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'securityscanningnotificationworker.py', 'pid': '76'} gunicorn-registry stdout | 2023-08-08 06:45:34,068 [207] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '207'} gunicorn-registry stdout | 2023-08-08 06:45:34,071 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '204'} gunicorn-registry stdout | 2023-08-08 06:45:34,074 [206] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '206'} gunicorn-registry stdout | 2023-08-08 06:45:34,077 [208] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '208'} gunicorn-registry stdout | 2023-08-08 06:45:34,088 [209] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '209'} gunicorn-registry stdout | 2023-08-08 06:45:34,091 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '63'} gunicorn-registry stdout | 2023-08-08 06:45:34,095 [203] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '203'} gunicorn-registry stdout | 2023-08-08 06:45:34,099 [210] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '210'} gunicorn-registry stdout | 2023-08-08 06:45:34,103 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'registry:application', 'pid': '205'} namespacegcworker stdout | 2023-08-08 06:45:34,257 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'namespacegcworker.py', 'pid': '69'} expiredappspecifictokenworker stdout | 2023-08-08 06:45:34,749 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '59'} notificationworker stdout | 2023-08-08 06:45:35,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:45:35,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:45.371682+00:00 (in 9.999491 seconds) notificationworker stdout | 2023-08-08 06:45:35,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:45 UTC)" (scheduled at 2023-08-08 06:45:35.371682+00:00) notificationworker stdout | 2023-08-08 06:45:35,372 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:45:35,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 35, 372650), True, datetime.datetime(2023, 8, 8, 6, 45, 35, 372650), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:45:35,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:45:35,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:45:35,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:45 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:45:36,580 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:45:36,580 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:54.393627+00:00 (in 17.813185 seconds) namespacegcworker stdout | 2023-08-08 06:45:36,580 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:36 UTC)" (scheduled at 2023-08-08 06:45:36.579841+00:00) namespacegcworker stdout | 2023-08-08 06:45:36,580 [69] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2023-08-08 06:45:36,581 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:36 UTC)" executed successfully builder stdout | 2023-08-08 06:45:37,106 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:45:37,115 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:45:37,115 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:45:37,119 [55] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:37,178 [198] [DEBUG] [app] Starting request: urn:request:5a165c1a-529f-4029-807e-5f57031d76d3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:45:37,178 [200] [DEBUG] [app] Starting request: urn:request:b0505fdd-5d87-40c6-b36d-df3000c0e6fd (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:45:37,180 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:37,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:37,182 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:37,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:45:37,183 [203] [DEBUG] [app] Starting request: urn:request:50e09159-b886-4eed-b11f-ccc2ba55061d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:45:37,184 [203] [DEBUG] [app] Ending request: urn:request:50e09159-b886-4eed-b11f-ccc2ba55061d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:50e09159-b886-4eed-b11f-ccc2ba55061d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:45:37,184 [210] [DEBUG] [app] Starting request: urn:request:a7b9944b-a68d-41bf-b3ee-1a5324fcfa4b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-registry stdout | 2023-08-08 06:45:37,184 [203] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:45:37,185 [210] [DEBUG] [app] Ending request: urn:request:a7b9944b-a68d-41bf-b3ee-1a5324fcfa4b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a7b9944b-a68d-41bf-b3ee-1a5324fcfa4b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:45:37,185 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2023-08-08 06:45:37,185 [210] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:37 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2023-08-08 06:45:37,185 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:37,186 [198] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:37,186 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:37,188 [198] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:37,189 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:37,189 [197] [DEBUG] [app] Starting request: urn:request:87bdcb7c-c357-4f2a-8733-0b667854f62d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:45:37,190 [200] [DEBUG] [app] Starting request: urn:request:97deb1d5-1408-4f09-a92c-e155ab11d978 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:45:37,190 [197] [DEBUG] [app] Ending request: urn:request:87bdcb7c-c357-4f2a-8733-0b667854f62d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:87bdcb7c-c357-4f2a-8733-0b667854f62d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:45:37,190 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:45:37,190 [200] [DEBUG] [app] Ending request: urn:request:97deb1d5-1408-4f09-a92c-e155ab11d978 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:97deb1d5-1408-4f09-a92c-e155ab11d978', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:45:37,190 [198] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:37 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:45:37,190 [200] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:37 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:45:37,191 [198] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:45:37,191 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:37,191 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:45:37,195 [198] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:45:37,195 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) buildlogsarchiver stdout | 2023-08-08 06:45:37,197 [56] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2023-08-08 06:45:37,198 [56] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:07.197412+00:00 (in 29.999554 seconds) buildlogsarchiver stdout | 2023-08-08 06:45:37,198 [56] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:46:07 UTC)" (scheduled at 2023-08-08 06:45:37.197412+00:00) buildlogsarchiver stdout | 2023-08-08 06:45:37,198 [56] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2023, 7, 24, 6, 45, 37, 198196), False, 50, 1, 0]) gunicorn-web stdout | 2023-08-08 06:45:37,201 [200] [DEBUG] [app] Ending request: urn:request:b0505fdd-5d87-40c6-b36d-df3000c0e6fd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b0505fdd-5d87-40c6-b36d-df3000c0e6fd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:45:37,201 [198] [DEBUG] [app] Ending request: urn:request:5a165c1a-529f-4029-807e-5f57031d76d3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5a165c1a-529f-4029-807e-5f57031d76d3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:45:37,201 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:37,201 [198] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:37,201 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:45:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:45:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:45:37 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.024) gunicorn-web stdout | 2023-08-08 06:45:37,201 [198] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:45:37 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" buildlogsarchiver stdout | 2023-08-08 06:45:37,204 [56] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id" FROM "repositorybuild" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) buildlogsarchiver stdout | 2023-08-08 06:45:37,205 [56] [DEBUG] [__main__] Archiving: d5b40e12-f92c-42e5-9c25-9e76946bc5a3 buildlogsarchiver stdout | 2023-08-08 06:45:37,206 [56] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2023-08-08 06:45:37,210 [56] [ERROR] [workers.worker] Operation raised exception buildlogsarchiver stdout | Traceback (most recent call last): buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func buildlogsarchiver stdout | return operation_func() buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs buildlogsarchiver stdout | log_archive.store_file( buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file buildlogsarchiver stdout | self._storage.stream_write( buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]] buildlogsarchiver stdout | KeyError: 'local_us' buildlogsarchiver stdout | 2023-08-08 06:45:37,210 [56] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-08-08 06:46:07 UTC)" executed successfully gcworker stdout | 2023-08-08 06:45:39,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:45:39,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:49.417660+00:00 (in 9.999437 seconds) gcworker stdout | 2023-08-08 06:45:39,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:49 UTC)" (scheduled at 2023-08-08 06:45:39.417660+00:00) gcworker stdout | 2023-08-08 06:45:39,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267539418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:45:39,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:45:39,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:45:39,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:49 UTC)" executed successfully chunkcleanupworker stdout | 2023-08-08 06:45:42,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:45:42,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:52.293757+00:00 (in 9.999480 seconds) chunkcleanupworker stdout | 2023-08-08 06:45:42,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:52 UTC)" (scheduled at 2023-08-08 06:45:42.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:45:42,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:45:42,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 42, 294731), True, datetime.datetime(2023, 8, 8, 6, 45, 42, 294731), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:45:42,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:45:42,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:45:42,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:52 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:45:45,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:45:45,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:55.371682+00:00 (in 9.999489 seconds) notificationworker stdout | 2023-08-08 06:45:45,372 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:55 UTC)" (scheduled at 2023-08-08 06:45:45.371682+00:00) notificationworker stdout | 2023-08-08 06:45:45,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:45:45,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 45, 373126), True, datetime.datetime(2023, 8, 8, 6, 45, 45, 373126), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:45:45,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:45:45,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:45:45,381 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:55 UTC)" executed successfully builder stdout | 2023-08-08 06:45:47,124 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:45:47,133 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:45:47,133 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:45:47,137 [55] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:45:47,398 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2023-08-08 06:45:47,399 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:25.514341+00:00 (in 38.115320 seconds) exportactionlogsworker stdout | 2023-08-08 06:45:47,399 [60] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:47 UTC)" (scheduled at 2023-08-08 06:45:47.398551+00:00) exportactionlogsworker stdout | 2023-08-08 06:45:47,399 [60] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2023-08-08 06:45:47,400 [60] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 47, 399398), True, datetime.datetime(2023, 8, 8, 6, 45, 47, 399398), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2023-08-08 06:45:47,406 [60] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2023-08-08 06:45:47,406 [60] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2023-08-08 06:45:47,406 [60] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:47 UTC)" executed successfully gcworker stdout | 2023-08-08 06:45:49,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:45:49,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:59.417660+00:00 (in 9.999509 seconds) gcworker stdout | 2023-08-08 06:45:49,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:59 UTC)" (scheduled at 2023-08-08 06:45:49.417660+00:00) gcworker stdout | 2023-08-08 06:45:49,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267549418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:45:49,427 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:45:49,427 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:45:49,427 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:45:59 UTC)" executed successfully securityworker stdout | 2023-08-08 06:45:49,687 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:45:49,688 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:45:56.890741+00:00 (in 7.202853 seconds) securityworker stdout | 2023-08-08 06:45:49,688 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:46:19 UTC)" (scheduled at 2023-08-08 06:45:49.687363+00:00) securityworker stdout | 2023-08-08 06:45:49,688 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:45:49,688 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:45:49,690 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:45:49,691 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:45:49,697 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:45:49,697 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:46:19 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:45:50,205 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'notificationworker.py', 'pid': '71'} logrotateworker stdout | 2023-08-08 06:45:51,150 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'logrotateworker.py', 'pid': '66'} gunicorn-web stdout | 2023-08-08 06:45:52,179 [200] [DEBUG] [app] Starting request: urn:request:3ee78c2f-bf90-48de-92b0-d6061943ea2a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:45:52,180 [197] [DEBUG] [app] Starting request: urn:request:621b9a90-345d-4d86-b8c5-97ed29c6d201 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2023-08-08 06:45:52,180 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:52,181 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:52,183 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:52,184 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-registry stdout | 2023-08-08 06:45:52,184 [207] [DEBUG] [app] Starting request: urn:request:0c6f8875-5a04-4b4d-adaf-34fc892fe3db (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:45:52,185 [204] [DEBUG] [app] Starting request: urn:request:1ca9fa96-45c2-462b-844e-a48eab15a5bc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2023-08-08 06:45:52,185 [207] [DEBUG] [app] Ending request: urn:request:0c6f8875-5a04-4b4d-adaf-34fc892fe3db (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0c6f8875-5a04-4b4d-adaf-34fc892fe3db', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2023-08-08 06:45:52,185 [204] [DEBUG] [app] Ending request: urn:request:1ca9fa96-45c2-462b-844e-a48eab15a5bc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1ca9fa96-45c2-462b-844e-a48eab15a5bc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'bdfd0f4f', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-registry stdout | 2023-08-08 06:45:52,185 [207] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2023-08-08 06:45:52,185 [204] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:52 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 162 0.002) gunicorn-web stdout | 2023-08-08 06:45:52,186 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:52,186 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:52,187 [200] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:52,187 [197] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2023-08-08 06:45:52,189 [197] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:52,190 [200] [WARNING] [py.warnings] /app/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings gunicorn-web stdout | warnings.warn( gunicorn-web stdout | 2023-08-08 06:45:52,191 [197] [DEBUG] [app] Starting request: urn:request:ebb288e3-dbba-4be8-b005-c57eb0114eb0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:45:52,191 [197] [DEBUG] [app] Ending request: urn:request:ebb288e3-dbba-4be8-b005-c57eb0114eb0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ebb288e3-dbba-4be8-b005-c57eb0114eb0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2023-08-08 06:45:52,191 [199] [DEBUG] [app] Starting request: urn:request:18292b8d-b6f9-4cf7-b667-ce8422b4e438 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2023-08-08 06:45:52,191 [197] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2023-08-08 06:45:52,192 [197] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:52,192 [199] [DEBUG] [app] Ending request: urn:request:18292b8d-b6f9-4cf7-b667-ce8422b4e438 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:18292b8d-b6f9-4cf7-b667-ce8422b4e438', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'https://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 (-) - - [08/Aug/2023:06:45:52 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2023-08-08 06:45:52,192 [199] [INFO] [gunicorn.access] 127.0.0.1 - - [08/Aug/2023:06:45:52 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2023-08-08 06:45:52,192 [197] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:45:52,192 [200] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2023-08-08 06:45:52,193 [200] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2023-08-08 06:45:52,197 [197] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:45:52,197 [200] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2023-08-08 06:45:52,202 [197] [DEBUG] [app] Ending request: urn:request:621b9a90-345d-4d86-b8c5-97ed29c6d201 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:621b9a90-345d-4d86-b8c5-97ed29c6d201', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:45:52,202 [197] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:52,202 [200] [DEBUG] [app] Ending request: urn:request:3ee78c2f-bf90-48de-92b0-d6061943ea2a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3ee78c2f-bf90-48de-92b0-d6061943ea2a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://10.129.2.33/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '1ddfbe32', 'user-agent': 'kube-probe/1.26'} gunicorn-web stdout | 2023-08-08 06:45:52,202 [200] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2023-08-08 06:45:52,202 [197] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:45:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:45:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.023 47 0.023) gunicorn-web stdout | 2023-08-08 06:45:52,202 [200] [INFO] [gunicorn.access] 10.129.2.1 - - [08/Aug/2023:06:45:52 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.26" nginx stdout | 10.129.2.1 (-) - - [08/Aug/2023:06:45:52 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.26" (0.024 47 0.025) chunkcleanupworker stdout | 2023-08-08 06:45:52,294 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2023-08-08 06:45:52,294 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:02.293757+00:00 (in 9.999508 seconds) chunkcleanupworker stdout | 2023-08-08 06:45:52,294 [57] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:46:02 UTC)" (scheduled at 2023-08-08 06:45:52.293757+00:00) chunkcleanupworker stdout | 2023-08-08 06:45:52,294 [57] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2023-08-08 06:45:52,295 [57] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 52, 294703), True, datetime.datetime(2023, 8, 8, 6, 45, 52, 294703), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2023-08-08 06:45:52,302 [57] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2023-08-08 06:45:52,302 [57] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2023-08-08 06:45:52,302 [57] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:46:02 UTC)" executed successfully manifestbackfillworker stdout | 2023-08-08 06:45:52,314 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'manifestbackfillworker.py', 'pid': '67'} storagereplication stdout | 2023-08-08 06:45:52,892 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'storagereplication.py', 'pid': '79'} securityscanningnotificationworker stdout | 2023-08-08 06:45:54,284 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2023-08-08 06:45:54,284 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:13.097375+00:00 (in 18.812744 seconds) securityscanningnotificationworker stdout | 2023-08-08 06:45:54,284 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:54 UTC)" (scheduled at 2023-08-08 06:45:54.284093+00:00) securityscanningnotificationworker stdout | 2023-08-08 06:45:54,284 [76] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2023-08-08 06:45:54,285 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:54 UTC)" executed successfully namespacegcworker stdout | 2023-08-08 06:45:54,394 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2023-08-08 06:45:54,394 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:36.579841+00:00 (in 42.185651 seconds) namespacegcworker stdout | 2023-08-08 06:45:54,394 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:54 UTC)" (scheduled at 2023-08-08 06:45:54.393627+00:00) namespacegcworker stdout | 2023-08-08 06:45:54,394 [69] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2023-08-08 06:45:54,395 [69] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 54, 394583), True, datetime.datetime(2023, 8, 8, 6, 45, 54, 394583), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2023-08-08 06:45:54,402 [69] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2023-08-08 06:45:54,402 [69] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2023-08-08 06:45:54,402 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2023-08-08 06:46:54 UTC)" executed successfully notificationworker stdout | 2023-08-08 06:45:55,372 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2023-08-08 06:45:55,372 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:02.178530+00:00 (in 6.805887 seconds) notificationworker stdout | 2023-08-08 06:45:55,373 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:46:05 UTC)" (scheduled at 2023-08-08 06:45:55.371682+00:00) notificationworker stdout | 2023-08-08 06:45:55,373 [71] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2023-08-08 06:45:55,373 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2023, 8, 8, 6, 45, 55, 373195), True, datetime.datetime(2023, 8, 8, 6, 45, 55, 373195), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2023-08-08 06:45:55,380 [71] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2023-08-08 06:45:55,380 [71] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2023-08-08 06:45:55,380 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2023-08-08 06:46:05 UTC)" executed successfully gcworker stdout | 2023-08-08 06:45:55,546 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'gcworker.py', 'pid': '61'} gunicorn-secscan stdout | 2023-08-08 06:45:55,684 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '64'} gunicorn-secscan stdout | 2023-08-08 06:45:55,688 [166] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '166'} gunicorn-secscan stdout | 2023-08-08 06:45:55,693 [167] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'secscan:application', 'pid': '167'} securityworker stdout | 2023-08-08 06:45:56,891 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2023-08-08 06:45:56,891 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:19.687363+00:00 (in 22.796130 seconds) securityworker stdout | 2023-08-08 06:45:56,891 [77] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:46:26 UTC)" (scheduled at 2023-08-08 06:45:56.890741+00:00) securityworker stdout | 2023-08-08 06:45:56,891 [77] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:45:56,891 [77] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2023-08-08 06:45:56,892 [77] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:45:56,892 [77] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2023-08-08 06:45:56,892 [77] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:45:56,892 [77] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2023-08-08 06:45:56,892 [77] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2023-08-08 06:45:56,894 [77] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2023-08-08 06:45:56,896 [77] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2023-08-08 06:45:56,902 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2023-08-08 06:45:56,904 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:56,904 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:56,904 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2023, 8, 8, 6, 40, 56, 896149), 1, 2]) securityworker stdout | 2023-08-08 06:45:56,905 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:56,906 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:56,906 [77] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '57d6b7657adde342d823a8c046d051c0', datetime.datetime(2023, 8, 8, 6, 40, 56, 896149), 1, 2]) securityworker stdout | 2023-08-08 06:45:56,907 [77] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2023-08-08 06:45:56,907 [77] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2023-08-08 06:45:56,907 [77] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:45:56,907 [77] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2023-08-08 06:45:56,908 [77] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2023-08-08 06:45:56,908 [77] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2023-08-08 06:45:56,908 [77] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2023-08-08 06:45:56,908 [77] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2023-08-08 06:45:56,908 [77] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2023-08-08 06:45:56,908 [77] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2023-08-08 06:46:26 UTC)" executed successfully builder stdout | 2023-08-08 06:45:57,146 [55] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2023-08-08 06:45:57,154 [55] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2023-08-08 06:45:57,154 [55] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2023-08-08 06:45:57,159 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2023-08-08 06:45:57,381 [54] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'blobuploadcleanupworker.py', 'pid': '54'} chunkcleanupworker stdout | 2023-08-08 06:45:57,727 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'chunkcleanupworker.py', 'pid': '57'} repositorygcworker stdout | 2023-08-08 06:45:57,766 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositorygcworker.py', 'pid': '75'} gunicorn-web stdout | 2023-08-08 06:45:58,069 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '65'} gunicorn-web stdout | 2023-08-08 06:45:58,430 [198] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '198'} gunicorn-web stdout | 2023-08-08 06:45:58,569 [199] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '199'} gunicorn-web stdout | 2023-08-08 06:45:58,572 [200] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '200'} gunicorn-web stdout | 2023-08-08 06:45:58,580 [197] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'web:application', 'pid': '197'} teamsyncworker stdout | 2023-08-08 06:45:58,780 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'teamsyncworker.py', 'pid': '80'} repositoryactioncounter stdout | 2023-08-08 06:45:59,210 [74] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'repositoryactioncounter.py', 'pid': '74'} gcworker stdout | 2023-08-08 06:45:59,418 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2023-08-08 06:45:59,418 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2023-08-08 06:46:09.417660+00:00 (in 9.999491 seconds) gcworker stdout | 2023-08-08 06:45:59,418 [61] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:46:09 UTC)" (scheduled at 2023-08-08 06:45:59.417660+00:00) gcworker stdout | 2023-08-08 06:45:59,419 [61] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1690267559418, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2023-08-08 06:45:59,428 [61] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2023-08-08 06:45:59,428 [61] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2023-08-08 06:45:59,428 [61] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:10], next run at: 2023-08-08 06:46:09 UTC)" executed successfully queuecleanupworker stdout | 2023-08-08 06:45:59,474 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-758d8cbbb5-6xw9b', 'process_name': 'queuecleanupworker.py', 'pid': '73'}